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Abstract 

The  design-build  method  for  construction  project  delivery  continues  to  grow  in 
both  the  private  and  public  sector.  Several  government  agencies  have  observed,  through 
experience  with  design-build,  positive  results  which  give  “anecdotal”  credibility  to 
design-build  methods.  The  objective  of  this  study  is  to  compare  the  performance  of  the 
design-build  delivery  method  with  traditional  design-bid-build  approaches  for  Air  Force 
(AF)  military  construction  (MILCON). 

Data  related  to  835  (278  design-build,  557  traditional)  MILCON  projects  were 
gathered  from  the  Automated  Civil  Engineer  System  -  Project  Management  Module 
(ACES-PM)  for  Fiscal  Years  1996-2006.  The  design-build  method  had  better 
performance  for  six  of  eight  metrics  with  highly  significant  results  for  cost  growth  and 
number  of  modifications  per  million  dollars.  The  traditional  method  experienced  a 
highly  significant  advantage  for  the  metrics  of  construction  timeline  and  total  project 
time.  The  historical  analysis  revealed  that  design-build  MILCON  has  improved 
significantly  for  cost  growth,  modifications  per  million  dollars,  construction  timeline,  and 
total  project  time.  The  traditional  method  also  improved  for  the  cost  growth  and 
modifications  per  million  dollars  metrics.  Finally,  the  facility  type  analysis  revealed  that 
the  design-build  method  was  best  suited  for  seven  of  the  nine  facility  types.  This  study 
provides  empirical  evidence  of  where  the  design-build  delivery  method  provides  an 
advantage  to  the  traditional  method  for  AF  MILCON  execution. 
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AN  ANALYSIS  OF  THE  DESIGN-BUILD  DELIVERY  APPROACH  IN  AIR 
FORCE  MILITARY  CONSTRUCTION 

I.  Introduction 

The  construction  industry  is  continually  trying  to  meet  the  demands  of  project 
cost,  quality,  and  time  (Chan,  2002).  The  construction  industry  has  a  large  impact  on  the 
national  economy  with  a  2005  Gross  Domestic  Product  (GDP)  for  new  non-residential 
construction  of  $441  billion  dollars.  Military  spending  greatly  impacts  the  construction 
industry.  The  2006  Military  Construction  (MILCON)  and  Veterans  Affairs 
Appropriations  contributed  $82.5  billion  in  appropriations  for  military  construction, 
military  family  housing,  and  veterans'  affairs  programs  (Johnson,  2005).  The  FY06 
MILCON  budget  was  $16.26B  (Kyle,  2006)  of  which  the  AF  portion  was  $1.07B 
(Department  of  the  Air  Force,  2005).  The  Department  of  Defense,  specifically  the  United 
States  Air  Force  (USAF),  faces  several  budgetary  constraints  in  maintaining 
infrastructure  and  modernizing  facilities:  1)  funding  major  military  operations  in 
Afghanistan  and  Iraq  for  the  Global  War  on  Terrorism,  2)  repositioning  thousands  of 
personnel  and  consolidating  assets  through  Base  Realignment  and  Closures  (BRAC) 

2005,  and  3)  implementing  Program  Budget  Decision  (PBD)  720  to  recapitalize  and 
modernize  aircraft  in  the  USAF  inventory  by  retiring  aircraft  and  reducing  40,000  active 
duty  and  17,000  air  reserve  component  member  numbers  (Eulberg,  2007).  Each  of  these 
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constraining  factors  highlights  the  need  for  a  construction  mechanism  that  can  create 
quality  products,  on  a  rapid  schedule,  and  within  budget.  This  study  will  determine  if  the 
design-build  delivery  method  for  MILCON  is  such  a  mechanism. 

Background 

The  Military  Construction  (MILCON)  program  provides  the  armed  forces  with 
new  facilities  and  major  renovations  that  cost  $750,000  or  more.  These  large  projects 
typically  take  between  four  to  five  years  to  go  from  a  user  defined  requirement  to  the 
completed  project  (Department  of  the  Air  Force,  2003).  The  Air  Force  (AF)  manages 
and  records  the  status  of  MILCON  projects  from  planning  through  construction  in  the 
Automated  Civil  Engineer  System  -  Project  Management  Module  (ACES-PM).  Data  in 
ACES-PM  is  reported  to  and  reviewed  by  Air  Force  Headquarters  staff  and  Congress  for 
insertion  into  the  President’s  Budget.  Funding  for  MILCON  projects  is  strictly 
monitored  and  approved  by  Congress  annually  through  the  Military  Construction 
Appropriations  Act.  Approved  MILCON  projects  have  five  years  to  be  completed  before 
the  appropriation  expires  (Department  of  the  Air  Force,  2003). 

Design-Bid-Build 

Historically,  MILCON  projects  have  been  built  using  the  design-bid-build 
(traditional)  delivery  method  (Department  of  the  Air  Force,  2000).  Design-bid-build  is 
defined  as: 

“the  project  delivery  approach  where  the  Owner  commissions  an  architect  or 
engineer  to  prepare  drawings  and  specifications  under  a  design  services  contract, 
and  separately  contracts  for  at-risk  construction,  by  engaging  a  contractor  through 
competitive  bidding  or  negotiation.”  (DBIA,  2007a) 
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Federal  laws  and  regulations  mandated  the  use  of  the  design-bid-build  method  for  the 
public  sector  (Beard  et  ah,  2001).  The  traditional  method  is  a  simple  process  to  manage 
that  is  well  understood  by  owners,  designers,  and  builders  (Department  of  the  Air  Force, 
2000).  This  method  appealed  to  owners  due  to  its  established  track  record,  the  complete 
control  over  project  design,  and  the  award  given  to  the  lowest  bidder  from  competitive 
bidding  (Webster,  1997;  Department  of  the  Air  Force,  2000;  Loulakis,  2003).  However, 
the  traditional  method  came  under  criticism  in  the  1970s  due  to  increases  in  claims, 
disputes,  and  project  delays  (Cushman  &  Loulakis,  2001).  The  traditional  method  placed 
the  owner  as  the  arbitrator  between  the  construction  contractor  and  the  designer.  AF 
project  managers  in  charge  of  MILCON  projects  were  “managing  by  change  order”  when 
low  bid  contractors  were  searching  for  design  errors  and  modifications  to  increase  profits 
(Langley,  2007b). 

Design-Build 

Creation  of  alternative  delivery  methods  resulted  from  the  inefficiencies  of  the 
traditional  methods  (Beard  et  ah,  2001;  Cushman  &  Loulakis,  2001).  The  most  popular 
alternative  method  is  design-build  which  has  steadily  increased  in  popularity  and 
performance  (DBIA,  2007b).  Design-build  is  defined  as: 

“a  system  of  contracting  under  which  one  entity  performs  both  architecture¬ 
engineering  and  construction  under  one  single  contract.”  (DBIA,  2007a) 

The  design-build  method  removed  the  owner  from  acting  as  a  middle  man  between  the 
designer  and  builder.  The  responsibility  for  errors  and  omissions,  faulty  performance, 
and  coordination  of  problems  now  rested  with  the  design-build  contractor  (Cushman  & 
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Loulakis,  2001;  Link,  2006).  This  resulted  in  the  reduction  of  claims  and  litigation, 
increased  time  and  cost  savings,  reduced  owner  administrative  burden,  and  higher  quality 
(Konchar  &  Sanvido,  1998;  Beard  et  ah,  2001;  Cushman  &  Loulakis,  2001;  Link,  2006). 

The  use  of  the  design-build  method  for  federal  construction  required  a  change  in 
procurement  law.  Only  a  few  pilot  projects  were  authorized  to  use  design-build  from 
1986  to  1996  (Department  of  the  Air  Force,  2000).  The  rapid  growth  in  the  use  of 
design-build  did  not  occur  until  the  Clinger-Cohen  Act  of  1996  authorized  the  unlimited 
use  of  design-build  for  federal  use  (Loulakis,  2003).  As  a  result  of  observed  successes 
and  the  approval  for  use,  design-build  looks  to  overtake  design-bid-build  as  the  premier 
project  delivery  method  (DBIA,  2007b). 

Motivation 

Few  formal,  in-depth  studies  have  been  conducted  to  validate  the  claim  that  the 
design-build  method  is  a  significantly  better  project  delivery  process  for  military 
construction.  Studies  and  reports  have  been  qualitative  and  given  anecdotal  support  for 
design-build  methods  (Mouritsen,  1993;  Webster,  1997;  Glardon,  2006).  The 
quantitative  MILCON  design-build  studies  that  were  accomplished  occurred  at  or  prior  to 
the  approval  of  design-build  for  all  projects.  These  studies  are  dated  and  are  in  need  of 
validation  and  comparison  to  determine  if  the  design-build  process  has  improved  over 
time  (Webster,  1997;  Konchar  &  Sanvido,  1998).  Additionally,  these  studies  were 
accomplished  when  the  design-build  process  was  new  to  most  project  managers  so  the 
learning  curve  was  steep  (Buckingham,  1989;  Mouritsen,  1993). 
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The  question  arises  as  to  whether  design-build  has  improved  over  time.  Is  the 
process  more  efficient  due  to  AF  project  manager’s  increased  familiarity  with  the 
process?  Or,  now  that  it  has  become  the  delivery  method  of  choice,  has  design-build 
become  the  status  quo  and  no  longer  produces  the  advantages  from  its  early  days? 

Problem  Statement 

The  goal  of  this  thesis  is  to  use  an  empirical  approach  to  assess  if  the  design-build 
delivery  method  is  better  than  the  traditional  design-bid-build  method  for  Air  Force 
MILCON  projects.  It  will  determine  if  the  design-build  method  performs  better  for 
certain  facility  types.  The  results  will  be  analyzed  to  determine  if  the  success  of  the 
design-build  method  has  improved  through  the  years. 

Research  Questions 

A  thorough  analysis  of  MILCON  project  data  will  focus  on  answering  the 
following  research  questions: 

1 .  Does  the  design-build  delivery  method  for  AF  MILCON  result  in  better  cost 
performance  characteristics  than  the  traditional  design-bid-build  approach? 

2.  Does  the  design-build  delivery  method  for  AF  MILCON  result  in  better  schedule 
performance  characteristics  than  the  traditional  design-bid-build  approach? 

3.  Does  the  design-build  delivery  method  for  AF  MILCON  result  in  fewer 
modifications  than  the  traditional  design-bid-build  approach? 
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4.  Has  the  design-build  delivery  method  shown  a  statistically  significant  increased 
performance  level  over  the  traditional  design-bid-build  with  regard  to  cost  and 
schedule  measures? 

5.  Using  these  measures  of  success,  has  the  design-build  delivery  method  improved 
over  recent  years  at  a  statistically  significant  level? 

6.  What  facility  types  make  the  design-build  method  a  better  option  over  the 
traditional  design-bid-build  approach? 

Methodology 

Project  performance  data  will  be  collected  from  ACES-PM  for  Fiscal  Year  (FY)  1996 
to  FY  2006.  Traditional  and  design-build  projects  will  be  selected  from  this  data 
according  to  the  following  criteria:  continental  United  States  locations,  minimum  project 
value  at  the  MILCON  spending  level,  construction  recorded  at  100%  complete,  and 
exclusion  of  military  family  housing  (MFH)  projects. 

An  investigation  of  previous  research  will  identify  the  perfonnance  metrics  used 
to  compare  project  delivery  methods.  Eight  performance  metrics  will  be  used  by  this 
study  and  include:  unit  cost,  cost  growth,  schedule  growth,  construction  speed, 
modifications  per  million  dollars,  C WE/PA  ratio,  construction  timeline,  and  total  project 
time.  The  study  will  detennine  which  perfonnance  metrics  have  shown  a  statistically 
significant  difference  between  the  design-bid-build  and  design-build  delivery  methods. 
For  this  study,  a  one-tailed  test  statistic  that  produces  a  p-value  less  than  0.05  will  be 
considered  statistically  significant  and  highly  significant  if  less  than  0.01  (Webster, 

1997). 
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The  selected  data  will  be  grouped  into  two-year  increments  and  compared  within  the 
study  to  detennine  if  the  design-build  method  has  improved  through  time.  Additionally, 
facilities  will  be  grouped  according  to  Category  Codes  in  order  to  identify  which  delivery 
method  has  an  advantage  when  applied  to  a  particular  facility  type. 

Assumptions  and  limitations 

This  study  compares  the  design-build  and  traditional  delivery  methods  from 
MILCON  data  obtained  from  ACES-PM.  This  research  was  conducted  with  the 
assumption  that,  because  of  the  oversight  of  ACES-PM  data  by  MAJCOMs,  HQ  Air 
Force,  and  Congress,  the  data  in  ACES-PM  is  accurate  and  current.  This  study 
experienced  limitations  due  to  the  scope  of  analysis  and  the  use  of  ACES-PM  data.  The 
scope  of  this  research  was  limited  to  conducting  a  strict  empirical  analysis  and  did  not 
investigate  causality  of  the  results. 

The  use  of  ACES-PM  data  placed  several  unavoidable  limitations  on  this  research 
that  must  be  identified.  MILCON  project  timelines  are  documented  from  the  notice  to 
proceed  (NTP)  date  to  the  beneficial  occupancy  date  (BOD).  The  NTP  to  BOD  field  for 
design-build  projects  included  design  and  construction  time  but  only  measure  the 
construction  time  for  traditional  projects.  As  a  result,  this  restriction  on  metrics  using 
NTP  to  BOD  skewed  the  results  of  in  favor  of  traditional  projects.  The  construction 
timeline  and  schedule  growth  metric  results  required  the  reader  to  interpret  the  findings 
with  an  understanding  that  NTP  to  BOD  for  design-build  projects  include  design  and 
construction  whereas  the  NTP  to  BOD  for  traditional  projects  only  include  construction. 
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Additionally,  ACES-PM  does  not  document  the  cause  of  modifications.  This  study  was 
limited  by  the  assumption  that  all  modifications  were  a  result  of  a  negative  cause. 

Implications 

The  results  of  this  study  could  be  used  by  Department  of  Defense  project 
managers  to  enhance  project  management  of  MILCON  projects.  The  results  will 
quantitatively  show  where  design-build  provides  an  advantage  to  the  design-bid-build 
method.  This  study  will  also  provide  project  managers  with  the  knowledge  of  which 
delivery  method  is  historically  more  efficient  for  a  certain  facility  type.  The  results  will 
finally  provide  AF  project  managers  the  answer  to  the  question  of  when  or  if  to  use  the 
design-build  method  for  MILCON  projects. 

Thesis  Organization 

Chapter  1  establishes  the  framework  for  the  study  by  describing  the  impacts  of 
federal  spending  on  the  construction  industry  and  the  budgetary  challenges  facing  the  Air 
Force.  The  MILCON  program  is  described  followed  by  the  primary  means  to  construct 
MILCON  projects:  the  traditional  design-bid-build  and  design-build  delivery  methods. 
The  problem  statement  and  research  questions  identify  the  focus  of  the  study  to 
determine  if  the  design-build  method  performs  better  than  the  design-bid-build  method. 
Chapter  2  examines  the  literature  on  MILCON,  project  procurement  and  delivery 
methods,  and  project  performance  measures.  It  establishes  a  foundation  for  the  research 
methodology  and  identifies  the  gaps  left  in  previous  studies  that  will  be  filled  by  this 
study.  Chapter  3  describes  the  methodology  used  to  quantitatively  compare  the  two 
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delivery  methods.  It  identifies  the  performance  metrics  used  in  this  study  and  how  the 
metrics  were  calculated.  The  perfonnance  metrics  will  be  compared  between  the  design- 
bid-build  and  design-build  method  to  determine  if  the  results  are  statistically  significant. 
Chapter  4  outlines  the  results  and  analysis  of  the  study  over  each  perfonnance  metric. 
Chapter  5  provides  the  discussion  and  conclusions  gained  from  the  study  along  with 
recommendations  for  future  research. 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  investigates  the  current  literature  and  research  regarding  military 
construction  (MILCON)  project  delivery  methods.  The  purpose  is  to  understand  the  key 
characteristics  of  the  MILCON  process,  project  delivery  methods,  and  project  success. 
This  literature  review  is  organized  in  the  following  manner:  design-build  history,  history 
of  MILCON  design-build,  MILCON  overview,  project  delivery  methods,  project 
selection  criteria,  project  delivery  vs.  procurement,  project  procurement  methods,  project 
success  criteria,  study  performance  metrics,  and  previous  studies.  Finally,  a  literature 
review  summary  will  identify  the  gaps  found  in  the  literature  that  this  study  will  address. 
The  literature  review  serves  as  the  foundation  for  the  methodology  used  to  compare 
delivery  methods. 

Design-Build  History 

Appointing  a  single  entity  in  charge  of  all  aspects  of  a  project  is  not  a  new 
concept.  The  design-build  concept  has  a  rich  history  descending  from  the  “master 
builders”  or  “master  masons”  of  ancient  Egypt,  Greece,  and  Rome  (Beard  et  ah,  2001; 
Cushman  &  Loulakis,  2001).  Master  builders  did  not  distinguish  a  project  between  its 
design  and  construction  phases.  They  coordinated  and  controlled  every  aspect  of  a 
project  including  material  procurement  and  selection,  project  design,  supervising 
craftsmen,  and  project  financing  (Cushman  &  Loulakis,  2001). 

The  Renaissance  brought  about  the  first  challenge  to  the  idea  of  a  master  builder 
at  the  time  of  the  most  famous  master  builder  Filippo  Brunelleschi  (1377-1446). 
Brunelleschi  was  commissioned  to  build  the  Gothic  Cathedral  of  Florence  in  1420  (Beard 
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et  al.,  2001).  The  master  builders  were  challenged  by  Leone  Batista  Alberti,  who 
believed  in  the  separation  of  design  and  construction,  when  he  published  the  first 
architectural  printed  work  De  re  aedificatoria  (On  Edifices)  in  1485  (Cushman  & 
Loulakis,  2001).  The  popular  view  of  a  master  builder  remained  in  the  majority  until  the 
industrial  revolution. 

Industrial  Revolution 

The  Industrial  Revolution  produced  advances  in  technology,  manufacturing,  and 
productivity  on  a  tremendous  scale.  The  development  of  specialties  arose  during  this 
time,  thereby  distinguishing  designers  from  builders.  Beard  et  al.  attributed  the  principle 
changes  in  the  organization  of  design  and  construction  to  the  following  five  factors 
(2001;  p.19): 

1 .  Task  specialization:  the  increase  in  complexity  of  industry  drove  the  requirement 
for  specialized  engineer  and  architect  expertise,  but  not  builders. 

2.  Ability  to  communicate  design  intent:  expanding  design  market  created 
standardized  systems  of  drawings  and  specifications  which  enabled  the  designers 
to  work  off  site  and  not  be  tied  to  specific  projects. 

3.  Division  of  labor:  dividing  work  into  individual  tasks  segregated  the  intellectual 
process  of  design  from  the  physical  act  of  construction. 

4.  Entrepreneurship:  Builders  routinely  worked  with  contractual  risk  in  order  for 
business  growth  while  designers  were  fundamentally  risk  averse. 

5.  Need  for  capital:  Builders  now  required  large  capital  from  nonparticipating 
owners  to  support  the  new  machinery  and  large  labor  force.  Designers  were 


11 


unable  to  partner  with  such  stockholders  due  to  the  “ethic  of  individual 
professional  responsibility”  (Beard  et  al;  2001;  p.20). 

Professional  Societies 

The  need  for  specialization,  from  increased  industry  complexity,  formed 
professional  societies  that  increased  the  separation  of  design  from  construction 
professions.  The  American  Society  of  Civil  Engineers  and  Architects  (ASCE)  were 
formed  in  1852.  The  need  for  a  separate  architect  society  in  1857  lead  to  the  American 
Institute  of  Architects  (AIA)  and  the  “and  Architects”  was  removed  from  ASCE.  Both 
societies  saw  the  importance  of  separating  the  design  professionals  from  the  building 
contractor  and  material  supplier  (Beard  et  al.,  2001). 

Twentieth  Century 

The  twentieth  century  brought  about  legal  regulation  of  construction  contracting 
practices.  In  order  to  increase  accountability  and  reduce  corruption,  the  courts  passed 
legislation  further  separating  design  from  construction.  The  Miller  Act  of  1935  protected 
the  federal  government  from  contractors  that  lacked  the  capital  to  finish  a  project  (Beard 
et  al.,  2001).  The  1947  Armed  Services  Procurement  Act  established  procurement 
procedures  for  the  military.  The  Federal  Property  and  Administrative  Procedures  Act  of 
1949  followed  for  civilian  agencies  and  federal  public  works.  Both  acts  directed  federal 
agencies  to  use  negotiation  procedures  for  architectural  engineering  (A-E)  services  and 
separate  competitive  bid  procedures  for  construction  (Loulakis,  2003).  A-E  services  and 
construction  acquisition  was  further  regulated  by  the  1972  Brooks  Act,  40  U.S.C.  SS  541. 
Construction  had  to  be  bid  competitively  where  A-E  acquisition  was  focused  on  firm 
selection  based  on  competence  and  qualification  (Mouritsen,  1993;  Loulakis,  2003). 
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The  low  bid  regulation  further  separated  the  designer  from  the  builder  as 
described  by  Beard  et  al.:  “Implied  in  these  statutes  is  the  role  of  the  design  professional 
as  the  agent  of  the  owner,  and  the  prohibition  of  the  designer  from  having  any  financial 
relationship  with  the  builder”  (2001,  p.22).  Public  agencies  were  required  to  hire  a 
design  firm  to  produce  a  100%  design,  advertise  the  design  and  receive  bids  from 
construction  contractors,  and  then  hire  the  “lowest  responsible  bidder”  to  build  to  project. 
This  project  delivery  method  became  known  as  the  design-bid-build  or  “traditional” 
method  and  was  the  prevalent  method  for  the  twentieth  century  (Beard  et  al.,  2001). 

The  traditional  method  came  under  intense  scrutiny  during  the  1960s  and  1970s 
(Cushman  &  Loulakis,  2001).  Coordination  between  designers  and  builders  was  needed 
to  complete  the  increasingly  technically  complex  buildings  of  the  late  twentieth  century. 
Public  owners  desired  a  single  point  of  responsibility  for  design  and  construction.  Public 
owners  were  responsible  for  coordination  between  the  two  and  assumed  all  responsibility 
for  the  design  (Beard  et  al.,  2001).  Increases  in  claims,  disputes,  and  project  delays  were 
a  result,  driving  public  owners  to  search  for  a  better  delivery  method  for  projects.  The 
search  for  a  new  delivery  method  returned  to  the  master  builder  concept  and  was  called 
design-build  (Cushman  &  Loulakis,  2001). 

Emergence  of  Public  Design-Build 

The  increased  growth  and  use  of  the  design-build  method  can  be  directly 
attributed  to  the  expanding  use  in  the  public  sector  (Cushman  &  Loulakis,  2001).  The 
federal  agencies  that  significantly  contribute  to  the  construction  industry  include  the 
Department  of  Defense  (DOD),  Veterans  Affairs  (VA),  Department  of  State,  General 
Services  Administration  (GSA),  Environmental  Protection  Agency  (EPA),  National 
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Aeronautics  and  Space  Administration  (NASA),  Federal  Highway  Administration, 

United  States  Postal  Service  (USPS),  and  the  Department  of  Energy  (DOE)  (Mouritsen, 
1993;  Webster,  1997) 

The  DOD  first  began  using  the  design-build  method  in  1967  for  Military  Family 
Housing  (MFH)  in  an  attempt  to  duplicate  the  private  sector’s  success  with  design-build. 
Thirty  MFH  projects  had  been  successfully  completed  by  1972,  encouraging  Congress  to 
allow  design-build  methods  to  be  used  for  each  service’s  MFH  program  (Mouritsen, 
1993).  The  early  1970s  saw  the  first  use  of  competitive  public  design-build  procurement 
in  the  education  arena  for  non-residential  construction  (Beard  et  ah,  2001).  The  success 
of  MFH  led  to  experimentation  using  design-build  for  other  construction  projects,  which 
will  be  explained  in  the  next  section. 

Due  to  the  increase  of  the  design-build  process,  the  AIA  fonned  the  1975  Design- 
Build-Bid  Task  Force  which  made  recommendations  to  clarify  the  roles  of  professionals 
for  the  new  delivery  system.  The  AIA  used  these  recommendations  to  publish  the  first 
version  of  standard  design-build  contract  documents  in  1985  which  were  later  re¬ 
published  in  1996  (Beard  et  ah,  2001).  As  with  AIA  and  ASCE,  a  professional 
association  was  formed  to  facilitate  standards  for  the  new  delivery  industry.  The  Design- 
Build  Institute  of  America  (DBIA)  was  fonned  in  1993  as  a  result  of  the  increased  use 
and  success  of  design-build  (Cushman  &  Loulakis,  2001).  Owners,  designers,  and 
contractors  needed  guidance  and  standard  in  order  to  use  the  design-build  delivery 
method  to  its  fullest  potential.  The  goals  of  DBIA  are  to  promote  the  growth  of  design- 
build  and  to  disseminate  the  best  design-build  practices  throughout  the  construction 
industry  (DBIA,  2007c). 
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Military  Construction  (MILCON)  Design-Build  History 

The  use  of  design-build  by  public  agencies  has  been  controlled  by  legislation. 
Several  key  pieces  of  procurement  law  enabled  the  growth  of  design-build  in  MILCON 
(Loulakis,  2003).  The  Army  and  Air  Force  had  little  experience  with  design-build  prior 
to  1986.  The  Army  was  the  first  to  test  the  two-step  turnkey  method  on  three  projects  in 
1982  (Fort  Drum  Headquarters,  Fort  Harrison  Gym,  Fort  Stewart  Fire  House)  and  the 
one-step  turnkey  method  on  two  projects  in  1984  (Fort  Bliss  Gym,  Fort  Stewart  Gym). 
The  One-step  and  Two-step  procurement  methods  will  be  described  in  the  next  section. 
The  Army  observed  that  both  methods  delivered  the  projects  earlier  than  the  traditional 
method  and  within  budget.  This  success  led  to  the  limited  approval  of  design-build  for 
use  by  the  branches  of  the  military  (Buckingham,  1989). 

The  Military  Construction  Authorization  Act  of  1986  authorized  the  use  of 
design-build  for  MILCON,  Military  Family  Housing,  and  Operations  and  Maintenance 
(O&M)  programs  (Department  of  the  Air  Force,  2000).  Each  branch  of  the  military  was 
allowed  to  use  one-step  procedures  to  execute  a  maximum  of  three  design-build  projects 
annually  until  1990  (Buckingham,  1989).  These  pilot  projects  were  used  to  detennine  if 
design-build  was  an  effective  and  fair  delivery  method  for  MILCON.  The  restriction 
placed  on  the  use  of  one-step  design-build  was  then  removed  under  Title  10  U.S.C., 
Section  2862  by  the  Congress  in  1992  (Webster,  1997). 

The  Clinger-Cohen  Act  of  1996  opened  up  the  use  of  design-build  for  federal  use 
by  authorizing  the  use  of  the  two-step  procurement  procedures  for  any  federal  project 
whenever  the  situation  merited  design-build  delivery  (Beard  et  ah,  2001;  Loulakis,  2003). 
The  Clinger-Cohen  Act  amended  the  Defense  Authorization  Act  of  1996  and  the  1996 
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Federal  Acquisition  Reform  Act  (FARA)  (Beard  et  al.,  2001).  Design-build  use  by  the 
federal  government,  specifically  in  MILCON,  began  to  grow  after  1996  since  the  services 
were  no  longer  restricted  to  three  projects  per  year  (Beard  et  al.,  2001;  Loulakis,  2003). 


MILCON  Overview 

The  Military  Construction  program  provides  the  armed  forces  with  new  facilities 
and  major  renovations.  This  section  will  define  MILCON  and  describe  the  Air  Force 
MILCON  process.  The  overview  will  also  identify  the  key  milestones  important  to  the 
analysis  of  the  MILCON  process  with  regard  to  project  delivery  methods. 

MILCON  Defined 

Air  Force  Instruction  (AFI)  32-1021  defines  MILCON  as: 

“any  construction,  development,  conversion,  or  extension  of  any  kind 
carried  out  with  respect  to  a  military  installation.  MILCON  includes 
construction  projects  for  all  types  of  buildings,  roads,  airfield  pavements, 
and  utility  systems  costing  $750,000  or  more”  (Department  of  the  Air 
Force,  2003;  p.21). 

Prior  to  2003,  new  construction  projects  costing  more  than  $500,000  were  considered  as 
MILCON  projects.  Funds  for  MILCON  projects  are  approved  by  Congress  through  the 
Military  Construction  Appropriations  Act  annually.  Approved  MILCON  projects  have 
five  years  to  be  completed  before  the  appropriation  expires  (Department  of  the  Air  Force, 
2003). 

The  MILCON  Process 

The  MILCON  process,  as  detailed  in  Figure  1,  consists  of  four  elements: 
planning,  programming,  design,  and  construction  (Department  of  the  Air  Force,  2000). 
This  process  outlines  the  design-bid-build  method  and  design-build  method  for  MILCON 
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projects.  First  the  design-bid-build  method  will  be  outlined.  The  description  of  the 
design-build  process  is  provided  later  in  this  chapter.  Only  the  key  products  and 
milestones  of  each  element  will  be  discussed  for  brevity  sake.  Project  managers  should 
reference  the  Project  Managers  Guide  for  Design  and  Construction  (Department  of  the 
Air  Force,  2000),  known  as  the  Blue  Book,  for  a  comprehensive  description  of  the  Air 
Force  MILCON  process  and  policy. 

Planning 

The  goals  of  MILCON  planning  are  to  identify  critical  facility  requirements  and 
decide  the  most  effective  and  economical  method  to  meet  those  requirements.  Planning 
begins  when  a  new  requirement  is  identified  by  the  user,  major  command  (MAJCOM),  or 
Headquarters  AF  Staff  (Air  Staff)  to  support  a  renovation,  new  mission,  or  unit 
relocation.  The  planning  phase  consists  of  three  key  actions:  development  of  the 
Requirements  Document  (RD),  preparation  of  the  DD  Form  1391,  and  the  Certificate  of 
Compliance  (Department  of  the  Air  Force,  2003). 

The  Requirements  Document  (RD)  contains  the  project  description,  functional, 
architectural,  technical  requirements,  project  site,  and  programmed  amount  (PA) 
(Department  of  the  Air  Force,  2000).  The  PA  outlines  the  costs  for  completing  the 
projects.  The  PA  includes  the  government  estimate  for  the  facility,  site  work,  utilities, 
demolition,  communications,  contingency,  supervision,  inspection,  and  overhead  (SIOH). 
MILCON  projects  costing  over  $2  million  must  undergo  an  economic  analysis  to  validate 
the  project  is  the  most  effective  way  to  meet  the  identified  requirement.  Once  the  project 
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is  found  to  meet  the  requirement,  the  DD  Form  1391  will  be  developed  (Department  of 
the  Air  Force,  2003). 

The  Base  Civil  Engineer  (BCE)  will  develop  the  DD  Form  1391  (1391),  the 
purpose  of  which  is  to  ensure  the  project  is  aligned  with  the  long  tenn  plans  of  the  base. 
The  full  DD  Fonn  1391  package  contains  the  project  PA,  cost  estimate  breakdown, 
scope,  description  of  work,  justification  (requirement,  current  situation,  impact  if  not 
provided),  deficiency  detailed  data  (D  )  sheet,  floor  plan,  site  plan,  location  plan,  and 
Certificate  of  Compliance  (Frailie,  2004).  The  1391  is  then  input  into  the  Automated 
Civil  Engineer  System  -  Project  Management  Module  (ACES-PM)  and  will  be  updated, 
approved,  and  verified  at  MAJCOM,  Air  Staff,  and  DoD  before  it  is  included  in  the 
President’s  Budget  submittal  for  Congressional  approval  (Department  of  the  Air  Force, 
2000). 

Environmental  planning  is  accomplished  by  completing  a  Certificate  of 
Compliance.  The  Certificate  of  Compliance  contains  32  items  addressing  environmental 
and  cultural  resources,  environmental  pennits,  project  site  conditions,  and  sustainable 
design  considerations  for  the  project.  The  installation  commander  signs  the  document 
certifying  all  actions  have  been  completed  before  the  project  can  be  included  in  the 
Future  Years  Defense  Program  (FYDP)  (Department  of  the  Air  Force,  2003). 
Programming 

“Programming  is  the  process  of  developing  and  obtaining  approval  and  funding 
for  Military  Construction  (MILCON)  projects”  (Department  of  the  Air  Force,  2003; 
p.21).  The  MILCON  project  moves  into  the  programming  phase  after  the  base  1391  has 
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been  input  into  ACES-PM  and  submitted  to  MAJCOM.  Key  products  from  the 
programming  process  are  the  Requirements  and  Management  Plan  (RAMP)  and  final  DD 
Form  1391.  The  RAMP  must  be  completed  by  the  time  the  MAJCOM  submits  the  1391 
to  Air  Staff  for  inclusion  in  the  MILCON  program  (Department  of  the  Air  Force,  2000). 

The  RAMP  consists  of  the  RD  and  the  Project  Management  Plan  (PMP).  The 
PMP  includes  the  following  key  strategic  decisions  for  the  project  (Department  of  the  Air 
Force,  2000): 

•  Project  delivery  method  (traditional,  design-build,  fast  track,  construction 
management) 

•  Project  procurement  type  (firm  fixed  price,  cost  plus,  letter  contract) 

•  In-house  vs.  Architect-Engineer  (A-E)  design 

•  List  of  all  project  team  member  names  and  organizations 

•  Project  risk,  scheduling,  and  packaging 

The  completed  RAMP  and  DD  Form  1391  for  each  base’s  projects  are  prioritized  and 
submitted  by  MAJCOM  to  Air  Staff  to  be  included  in  the  MILCON  budget.  The  AF 
Program  Objective  Memorandum  (POM)  outlines  the  planned  AF  budget  and  includes 
the  FYDP.  The  FYDP  is  the  six-year  MILCON  program.  From  the  FYDP,  the  AF 
submits  a  two  fiscal  year  MILCON  budget  to  the  Office  of  the  Secretary  of  Defense 
(OSD)  and  Congress.  The  first  year  of  this  MILCON  budget  is  included  in  the 
President’s  Budget  (PB)  submitted  to  Congress.  Congress  then  approves  the  National 
Defense  Authorization  Act  and  the  Military  Construction  Appropriations  Act.  The 
authorization  and  appropriations  act  become  law  after  the  President  signs  both. 
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Construction  funding  can  then  be  arranged  after  the  bill  is  signed  by  the  President 
(Department  of  the  Air  Force,  2003). 

Design 

Much  of  the  programming  and  design  timelines  overlap  to  ensure  the  MILCON 
project  is  ready  for  construction  once  the  bill  is  signed.  The  Air  Staff  Planning 
Instruction  (PI)  directs  the  MAJCOM  to  change  the  project  from  programming  status  to 
design  status.  The  Design  Instruction  (DI)  is  directed  from  the  MAJCOM  to  the  design 
agent  to  initiate  design  of  the  project  in  accordance  with  the  RAMP  and  DD  Form  1391. 
The  Air  Force  is  not  its  own  design  and  construction  agent.  The  AF  is  allowed  to  manage 
only  5%  of  MILCON  projects  at  the  Air  Force  Center  for  Engineering  and  the 
Environment  (AFCEE).  Typically,  the  US  Army  Corps  of  Engineers  (USAGE)  or  the 
Navy  Facilities  Engineering  Command  (NAVFAC)  acts  as  the  design  and  construction 
agent  for  the  Air  Force  (Thomburn,  1994).  Table  1  outlines  the  stages  for  MILCON 
design  for  the  traditional  delivery  method.  MILCON  design  produces  the  project 
drawings  and  specifications  needed  to  construct  the  facility  and  refines  the  cost  and  scope 
of  the  project  (Buckingham,  1989). 
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Table  1.  Authority  to  Proceed  Design  Stages 


(Department  of  I 

the  Air  Force,  2000;  p.2-7) 

Design  Stage 

%  Design 

Activity 

Selection  of  A-E 

2% 

■  Activities  up  to  award  of  design  contract 

Notice  to  Proceed  (NTP) 

3% 

■  Direct  A-E  to  initiate  design 

Project  Definition  (PD) 

15% 

■  Validate  project  requirements  outlined  in  RD 

■  Resolves  all  scope,  requirements,  and  cost 
differences 

Early  Preliminary  Design 

30% 

■  Design  review  including  drawing  and 
specification  submittals 

Preliminary  Design 

60% 

■  Design  review  including  drawing  and 
specification  submittals 

Pre-Final  Design 

90% 

■  Completion  of  design  and  preparation  of  invitation 
for  bids  (IFB)  (traditional  delivery) 

■  Completion  of  Request  for  Proposal  (RFP) 
documents  (design-build) 

The  time  allowed  for  design  is  determined  by  the  PA.  The  design  phase  begins 
with  the  issuing  of  the  PI  to  when  the  project  is  ready  to  advertise  (RTA).  Table  2  shows 
the  historic  observed  time  required  for  key  design  milestones  depending  on  project  PA 
(Department  of  the  Air  Force,  2000). 


Table  2.  Historical  Average  Design  Period 
(Department  of  the  Air  Force,  2000;  Table  2-1) 


If  PA  is: 

<$5M 

$5-1 0M 

>$10M 

Then  time  isfrom  AF  PI  to... 

NTP  PD  RTA 

4  months 

6  months 

11  months 

6  months 

9  months 

15  months 

7  months 

10  months 

17  months 

Construction 

The  construction  phase  of  the  MILCON  project  includes  the  solicitation  for  bids 
from  construction  contractors,  project  management  of  construction,  conducting  pre-final 
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and  final  inspections,  facility  acceptance/Beneficial  Occupancy  Date  (BOD),  and 
financial  close  out  (Department  of  the  Air  Force,  2000).  The  authority  to  advertise  a 
MILCON  project  by  the  MAJCOM  is  granted  by  Air  Staff  when  all  of  the  following 
criteria  are  accomplished  (Department  of  the  Air  Force,  2000;  p.6-5): 

•  Project  included  in  bills  signed  by  the  President 

•  Project  at  least  95%  designed  as  reported  in  ACES-PM 

•  Current  Working  Estimate  (CWE)/PA  ratio  is  not  greater  than  1 10% 

•  MAJCOM  fiscal  year  MILCON  program  CWE/PA  ratio  below  100% 

•  Environmental  Impact  Analysis  Process  (EIAP)  is  completed  and  reported  in 
ACES-PM 

The  contract  is  awarded  in  accordance  with  the  selected  procurement  method  criteria 
if  the  award  CWE  is  within  the  authorization  threshold.  The  contract  CWE  must  not 
exceed  the  PA  by  25%  or  $2.0  million.  The  project  must  be  redesigned,  re-bid,  or 
reprogrammed  if  the  CWE  is  above  the  threshold  (Department  of  the  Air  Force,  2000). 

The  time  for  MILCON  construction  contracts  has  been  based  off  of  Dirtkicker 
Criteria.  The  Dirtkicker  Award  program  was  established  to  provide  metrics  to  measure 
MILCON  execution  (Robbins,  2003).  Further  details  of  the  Dirtkicker  program  will  be 
discussed  later  in  this  study.  As  with  design,  MILCON  construction  timelines  are 
dependent  on  PA  (Table  3).  The  MILCON  project  is  complete  when  all  deficiencies 
have  been  corrected,  project  as-built  drawings  have  been  submitted,  all  costs  are 
recorded,  and  the  financial  close-out  has  been  accomplished  (Department  of  the  Air 
Force,  2000). 
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Table  3.  Dirtkicker  Construction  Timeline  (Fox,  2006) 


Then  time  from  NTP  to... 

If  PA  is: 

BOD 

<$5M 

365  days 

$5-20M 

540  days 

>$20M 

730  days 

The  MILCON  process  described  above  was  developed  from  the  vast  experience 
using  the  traditional  design-bid-build  method  for  MILCON  execution.  The  Air  Force, 
and  other  branches,  is  no  longer  limited  to  solely  using  the  traditional  method  for 
MILCON  construction  (Department  of  the  Air  Force,  2000;  Beard  et  ah,  2001).  The  next 
section  describes  the  project  delivery  methods  now  available  for  construction  projects. 

Project  Delivery  Methods 

Project  delivery  methods  are  the  processes  in  which  a  project  is  planned, 
designed,  and  constructed  (Beard  et  ah,  2001).  This  section  identifies  the  methods  used 
by  the  construction  industry  to  complete  projects.  First,  the  traditional  design-bid-build 
method  is  described.  A  brief  description  of  alternative  methods  will  identify  other 
options  for  construction  execution.  This  section  will  be  concluded  by  a  comprehensive 
discussion  of  the  design-build  method  and  its  application  to  MILCON. 

Traditional  Design-Bid-Build  Method 

The  traditional  delivery  method  used  by  the  Air  Force  is  design-bid-build. 

Broadly  summarizing,  the  government  hires  an  Architect-Engineer  (A-E)  firm  to  produce 
a  100%  design  of  the  project  with  all  drawings,  specifications,  and  contract  documents. 
This  design  is  then  advertised  through  an  Invitation  for  Bids  (IFB)  and  receives  bids  from 
construction  firms.  Typically,  the  “low  responsible  and  responsive  bidder”  (Department 
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of  the  Air  Force,  2000;  C.8  p.5)  is  awarded  the  contract  with  a  firm  fixed  price.  The 
construction  firm  then  completes  the  project  with  oversight  from  the  owner  (Department 
of  the  Air  Force,  2000). 

The  traditional  process  has  historically  been  the  primary  method  for  construction 
by  the  Department  of  Defense.  The  process  is  well  understood  by  project  managers 
because  all  the  steps,  processes,  requirement,  and  roles  have  been  codified  by 
professional  societies,  the  government,  designers,  and  contractors  (Department  of  the  Air 
Force,  2000).  Table  4  outlines  the  advantages  of  the  traditional  approach  identified 
through  previous  studies,  literature,  and  interviews. 


Table  4.  Advantages  of  Traditional  Approach 


Advantages 

Description 

Source 

Established  way  of  doing 
projects 

Well-established  legal  and  contractual  precedents 

2,5 

Appropriateness  for  competitive 
bidding 

Competitive  nature  of  bidding  obtains  lowest  price  for  construction 
based  on  100%  design  documents 

2,  5,7 

Complete  control  over  design 

Owner  holds  meetings  typically  at  30%,  60%  and  90%  design 
complete  stages  to  comment  on  all  drawings  and  specifications 

Low  price  award 

Owners  award  the  project  to  the  contractor  who  presented  the  lowest 
bid  price 

A/E  working  directly  for  the 

owner 

A-E  gives  professional  design  advice  to  the  owner  in  a  not-at-risk 
relationship 

5 

No  legal  barriers  to  procurement 
and  licensing 

Established  legal  findings  for  allocating  risk  and  responsibility. 
Established  procedure  for  licensing  A/Es  and  construction  linns  in  all 
states. 

5 

Contractor  assumes  all 

construction  risks 

Absorbs  weather  costs,  labor  disputes,  material  cost  increases,  and 
external  factors 

2 

Projects  is  fully  defined 

Design  phase  produces  100%  complete  drawings,  specifications,  and 
cost  estimates 

2,6 

Objective  contract  award 

Sealed  bid  packages  ensures  contract  award  based  on  price  and  not 
subjective  metrics 

7 

Key: 

1  Mouritsen  (1993)  5  Beard  et  al.  (2001) 

2  Fee  (1996)  6  Cushman  &  Loulakis  (2001) 

3  Webster  (1997)  7  Link  (2006) 

4  Department  of  the  Air  Force  (2000)  8  Langley  (2007b) 
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Several  problems  emerged  though  the  years  resulting  from  the  separation  of 
design  and  construction.  As  previously  stated,  increased  claims  and  litigation  along  with 
rising  costs  and  delayed  project  timelines  drove  owners  to  investigate  alternative  delivery 
methods.  Table  5  identifies  several  of  the  disadvantages  found  in  the  traditional  design- 
bid-build  delivery  method. 


Table  5.  Disadvantages  of  Traditional  Approach 


Disadvantages 

Description 

Source 

Owner  is  the  arbiter  between 
designer  and  constructor 

Owner  bears  the  risk  for  adequacy  of  design.  Designer  and  constructor 
disagreements  must  be  solved  by  the  owner 

5,6,7 

Owner  pays  for  changes 

Owner  hinds  change  orders  to  overcome  design  conflicts  and  change 
orders.  Increase  costs  deplete  contingency  hinds  and  could  lead  to 
litigation 

6,8 

No  shared  vision  or  goals  for 
between  the  owner,  designer,  and 
contractor 

Neither  party  is  totally  focused  on  the  ultimate  goals  of  the  project  or 
owner.  Designer  goals  focus  on  accuracy  and  quality  of  physical 
products.  Constructor  focuses  on  cost  and  schedule  management 

An  initial  low  bid  does  not 
necessarily  result  in  final  best 
value 

Preoccupation  with  low  first  cost  ignores  importance  of  past 
perfonnance,  good  enviromnental  practices,  concern  for  life  cycle 
performance,  and  other  best-value  selection  criteria. 

5,  6 

Price  not  certain  until  construction 

bid  is  received 

Bids  over  budget  present  problems  for  owners.  Cost  is  unknown  until 
the  100%  design  package  is  bid  on.  If  bids  are  over  the  owner's  budget 
the  project  must  be  redesigned  or  lowered  in  scope 

5,7 

Constructor  is  not  involved  in  the 
design 

Separation  of  designer  and  builder  is  required  by  the  traditional  process. 
Constructability  is  lowered  by  lack  of  construction  input  in  design. 

2,  5,  6,  7 

Design-Bid-Build  is  slower  than 
other  delivery  systems 

Linear  structure  includes  time  required  to  select  an  A-E,  design  to  100%, 
bid  the  design  package,  then  build  with  no  overlap 

2,5,6 

History  of  litigation 

Increased  disputes  between  the  constructor  and  designer  over  design 
clarity,  errors,  omissions,  in  place  construction  quality,  time  delays,  and 
other  project  related  issues 

2,5 

Change  orders 

Contractors  can  low  bid  and  recover  profits  by  generating  change  orders 
resulting  from  design  omission  and  errors. 

2,8 

Agency  may  need  more  technical 
staff 

Architects,  engineers,  and  construction  inspectors  typically  required  to 
review  drawings,  specifications,  and  inspect  construction 

7 

Key: 

1  Mouritsen  (1993)  5  Beard  et  al.  (2001) 

2  Fee  (1996)  6  Cushman  &  Loulakis  (2001) 

3  Webster  (1997)  7  Link  (2006) 

4  Department  of  the  Air  Force  (2000)  8  Langley  (2007b) 
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Alternative  Delivery  Methods 

Various  alternative  approaches  have  been  developed  in  order  to  meet  the  growing 
demands  of  owners  for  better  quality,  faster  project  delivery,  lower  cost,  and  less  risk 
(Pocock,  1996).  Most  alternative  project  approaches  center  on  the  concept  of  merging 
the  efforts  of  the  A-E  firm  and  construction  firm  together  to  increase  communication, 
accountability,  and  improve  the  design.  Other  alternative  procurement  methods  include: 

•  Bridging:  Unlike  true  design-build,  the  owner  develops  initial  designs  and 
specifications,  typically  30%  to  50%  complete  (Webster,  1997),  before 
contracting  with  a  design-build  firm.  Bridging  provides  owners  with  more 
control  over  design  but  can  limit  the  advantages  of  true  design-build. 

Bridging  is  not  recognized  by  the  DBIA  as  a  design-build  method  due  to  the 
level  of  design  completed  before  the  issuing  to  the  RFP  (Webster,  1997,  Beard 
et  ah,  2001). 

•  Fast  track:  Fast-track  is  similar  contractually  to  design-build.  However,  this 
method  expedites  construction  by  beginning  construction  before  working 
drawings  and  specifications  are  completed  (Webster,  1997;  Department  of  the 
Air  Force,  2000). 

•  Construction  management  (CM):  Fhe  owner  hires  a  designer  under  a  separate 
contract.  Before  the  final  design  is  complete,  a  construction  agent  is  hired  to 
review  the  design  for  constructability.  Fhe  construction  agent  is  then  hired  to 
take  the  project  into  construction  (CM  at  Risk).  In  CM  at  fee,  the  construction 
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agent  serves  solely  in  a  consulting  role  and  does  not  continue  into  construction 
(Webster,  1997;  Konchar  &  Sanvido,  1998;  Beard  et  al.,  2001). 

•  Partnering:  This  method  attempts  to  establish  long-tenn  relationships  between 
owners,  designers,  contractors  to  meet  mutually  beneficial  goals  (Pocock, 

1996;  Webster,  1997). 

•  Constructability:  This  method  emphasizes  bringing  in  a  construction  expert  to 
contribute  at  the  early  phases  of  planning.  This  concept  can  be  used  along 
with  other  delivery  methods  as  well  (Pocock,  1996). 

•  Turn-key:  Owner  develops  a  narrative  project  description  and  uses  it  in  an 
RFP  without  drawings  or  specifications  to  hire  a  design-builder  (Thomburn, 
1994).  The  design-build  firm  performs  additional  services  such  as  financing 
the  project,  land  acquisition,  obtaining  pennits,  and  maintaining  the  facility 
after  completion  (Webster,  1997;  Department  of  the  Air  Force,  2000). 

The  differences  between  the  traditional  method  and  alternative  delivery  methods 
are  displayed  in  Figure  2.  A  key  differentiation  between  the  delivery  methods  is  the  point 
at  which  a  firm  cost  is  known  for  the  project.  The  design- build  method  benefits  the 
owner  by  determining  the  known  contract  cost  earlier  in  the  project  timeline.  Therefore, 
the  budget  risk  of  the  owners  is  transferred  into  the  assumed  cost  risk  of  the  design-build 
contractor.  The  contractor  must  work  to  stay  within  the  budget  set  early  in  the  timeline. 
The  next  section  describes  the  alternative  delivery  method  this  study  will  focus  on: 
design-build. 
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"Classical"  Project 
Delivery 

(Design-Builder  warrants  to  Owner 
that  design  documents  are  complete 
and  free  (ram  error) 


"Traditional"  Project 
Delivery 

(Owner  warrants  to  Contractor 
that  design  documents  are  complete 
and  tree  from  error) 


Negotiated 
Selection  Design- 
Build 


Competitive 
Selection  Design- 
Build 


Design-Bid-Build 


Fee-Type  Const. 
Management 


Figure  2.  “Classical”  vs.  “Traditional”  project  delivery 
(Beard  et  aL,  2001;  Figure  9.1) 


Design-Build 

The  Design-Build  Institute  of  America  defines  design-build  as:  “a  system  of 
contracting  under  which  one  entity  performs  both  architecture/engineering  and 
construction  under  one  single  contract”  (DBIA,  2007a).  Design-build  is  now  the  second 
most  used  delivery  method  in  the  United  States  as  seen  in  Figure  3  (Department  of  the 
Air  Force,  2000,  DBIA,  2007b).  The  previously  skeptical  public  sector  has  dramatically 
increased  its  use  of  design-build  since  1998  (Loulakis,  2003).  The  DBIA  predicts  the 
design-build  method  will  overtake  the  traditional  method  in  the  next  ten  years. 
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Non-Residential  Design  and  Construction 
in  the  United  States 


Figure  3.  Market  Saturation  of  Delivery  Method  (DBIA,  2007b) 


The  popularity  for  design-build  stems  from  the  advantages  it  has  over  the 
traditional  design-bid-build  method  (Department  of  the  Air  Force,  2000).  Table  6 
displays  the  advantages  observed  in  the  literature.  As  previously  mentioned,  owners 
were  searching  for  a  delivery  mechanism  that  would  improve  upon  the  problems 
experienced  with  design-bid-build.  Owners  were  burdened  by  acting  as  an  arbitrator 
between  the  construction  contractor  and  the  designer  and  being  financially  responsible 
for  changes  and  errors  (Cushman  &  Loulakis,  2001;  Loulakis,  2003).  The  single  contract 
for  design  and  construction,  and  the  resulting  benefits,  is  what  draws  project  managers 
towards  design-build. 


30 


Table  6.  Advantages  of  Design-Build 


Advantages 

Description 

Source 

Single  source  of  responsibility  for 
design  and  construction 

Responsibility  for  errors  and  omissions,  faulty  performance,  coordination 
of  problems  lies  with  the  design-builder  instead  of  the  owner. 

1,2,  5,  6,  7,  8 

Time  savings 

Design-build  eliminates  the  bidding  periods  and  redesign  of  the 
traditional  method.  Materials  and  equipment  procurement,  and  site 
staging  can  begin  before  completed  design  documents. 

1,2,5,  6,7,8 

Early  knowledge  of  firm  costs 

Guaranteed  project  costs  are  known  at  proposer  selection.  Additional 
costs  savings  result  from  one  entity  coordinating  cost  estimates  for 
construction  as  designs  are  completed. 

4,  5,  6,  7 

Higher  quality 

The  design-build  contractor  is  responsible  for  the  entire  project. 
Construction  input  is  given  from  the  beginning  of  design  from  the 
builder.  Design  errors,  omissions,  and  defects  are  identified  and  quickly 
solved  from  within. 

2,  3,  5,  6,  7 

Cost-effectiveness 

Value  engineering  and  constructability  are  ongoing  throughout  the  whole 
process  resulting  in  lower  cost. 

1,4,  5,6 

Encourages  innovation 

Design-build  is  a  performance  based  system  instead  of  the  specifications 
based  traditional  method.  The  RFP  outlines  the  performance 
requirements  of  the  owner  and  the  proposers  may  use  different  solutions 
to  meet  the  owner's  goals. 

1,2,  4,5,7 

Lower  claims  and  litigation 

Owners  avoid  the  majority  of  claims  and  litigations  due  to  the 
responsibility  for  the  designs  rests  with  the  design-build  contractor.  The 
number  of  disputes  is  far  fewer  since  the  design-builder  has  no  one  to 
blame  for  errors  but  itself. 

1,4,  5,  7,  8 

Reduced  administrative  burden 

Does  not  require  the  many  architects,  engineers,  and  construction 
inspectors  typically  required  for  oversee  the  traditional  method. 

Personnel  required  to  administer  conflicts  between  contractors  has  been 
reduced. 

1,4,  6,7 

Key: 

1  Mouritsen  (1993) 

2  Fee (1996) 

3  Webster  (1997) 

4  Department  of  the  Air  Force  (2000) 


5  Beard  et  al.  (2001) 

6  Cushman  &  Loulakis  (2001) 

7  Link  (2006) 

8  Langley  (2007b) 


The  increased  popularity  and  observed  successes  of  design-build  have  produced 
several  issues.  The  disadvantages  listed  in  Table  7  predominantly  result  from  the  misuse 
of  the  method  by  owners,  agents,  and  firms  (Cushman  &  Loulakis,  2001).  Several 
owners  feel  that  design-build  can  be  used  for  any  project  and  fail  to  consider  other  project 
characteristics  that  also  contribute  to  project  success.  Additional  issues  arise  when  there 
is  a  lack  of  trust  between  the  owner  and  the  design-build  contractor  (Cushman  & 

Loulakis,  2001).  Some  owners  attempt  to  protect  themselves  by  placing  all  project  risk 
on  the  design-builders.  Other  owners  are  unwilling  to  release  the  creative  control  and 
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over  constrain  the  design-build  team  with  detailed  specifications  based  RFPs  instead  of 
performance  based  requirements  (Beard  et  al.,  2001;  Cushman  &  Loulakis,  2001; 
Loulakis,  2003). 


1 

"able  7.  Disadvantages  of  Design-Build 

Disadvantages 

Description 

Source 

Unfamiliarity  with  the  process 

Owners  and  practitioners  might  not  have  used  design-build  and  are 
unaccustomed  to  the  collaborative  method. 

5,6 

Experience  of  management  team 

Success  increases  as  the  team  gains  in  experience.  Inexperienced  teams 
might  need  to  hire  and  experienced  professional  owner's  representative 

4,  7,  8 

Adequate  owner  staffing 

An  owner  that  does  not  have  staff  to  adequately  develop  the  RFP  will 
have  difficulty  defining  and  presenting  their  needs  to  the  design-build 
team 

7 

Communicating  owner's  needs  in 
design-build  is  different 

The  owner's  performance  requirements  must  be  outlined  as  criteria  for 
design  and  not  detailed  specifications.  Owners  comfortable  with  the 
traditional  method  will  struggle  with  the  qualifications  based  proposals 
for  the  RFP  instead  of  the  low  cost  bid  on  complete  design  documents 

5 

Barriers  in  procurement  and 
licensing  laws 

Some  states  still  require  the  use  of  separate  design  and  construction 
contracts 

5 

Availability  of  insurance  and 
bonding  products  for  design-build 

Industry  still  wary  of  providing  the  same  coverage  to  design-build  firms 
as  traditional  construction  firms  resulting  in  higher  premiums  for 
insurance  and  bonding. 

1,2,5 

Trust  and  Control 

Inexperienced  owners  continue  to  desire  the  same  involved  design 
review  process  or  place  all  the  project  risk  on  the  contractor.  These 
owners  do  not  possess  the  trust  required  reduce  the  effectiveness, 
increase  cost,  and  delay  the  process. 

6,8 

Loss  of  designer  as  the  owner's 
advocate 

Both  the  designer  and  constructor  are  in  the  business  to  generate  profit. 
The  designer's  interests  are  no  longer  directly  tied  owner's  needs  as  in  the 
traditional  method 

1,2,7 

Subjective  contract  award 

The  process  may  bypass  the  competitive  bidding  process,  possibly  not 
affording  the  owner  the  lowest  price 

7 

Key: 

1  Mouritsen  (1993)  5  Beard  et  al.  (2001) 

2  Fee  (1996)  6  Cushman  &  Loulakis  (2001) 

3  Webster  (1997)  7  Link  (2006) 

4  Department  of  the  Air  Force  (2000)  8  Langley  (2007b) 


One  factor  that  detennines  whether  design-build  will  be  successful  for  a  project  is 
knowing  the  design-build  process  for  one’s  agency.  The  next  section  outlines  the  AF 
MILCON  design-build  process.  The  AF  MILCON  design-build  process  must  be 
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understood  in  order  to  know  what  type  of  MILCON  project  the  design-build  method 
should  be  used. 

MILCON  Design-Build  Process 

The  MILCON  design-build  process,  as  displayed  in  Figure  1,  can  be  summarized 
in  the  following  six  phases:  planning,  pre-design  activities  develop  RFP,  administer  RFP, 
proposal  evaluation  and  award,  and  design/construction  (Thornbum,  1994). 

1.  Planning:  The  decision,  by  the  responsible  MAJCOM,  to  use  the  design-build 
method  occurs  during  this  phase.  The  products  of  this  phase  are  the  RD  and  draft 
PMP  development  for  the  RAMP  and  sent  to  the  Design  Agent  (Thornbum, 

1994). 

2.  Pre-Design  Activities:  This  phase  develops  the  project  definition  (PD)  document 
which  will  guide  the  creation  of  the  RFP.  If  the  Design  Agent  is  unable  to 
develop  the  PD  or  RFP  in-house  due  to  inadequate  staff  or  experience,  an  A-E 
will  be  selected  to  develop  the  RFP  for  the  Agent  (Thornbum,  1994). 

3.  Develop  RFP:  RFP  development  is  the  most  critical  and  owner  resource  intensive 
phase  of  design-build  and  presents  the  greatest  departure  from  the  traditional 
design-bid-build  project  development  process  (Beard  et  ah,  2001).  The  RFP 
“must  clearly  describe  the  technical  requirements  of  the  project,  and  the  criteria 
for  evaluating  proposals,  and  the  contractual  relationship  between  the 
Government  and  Offeror”  (Department  of  the  Air  Force,  2000;  p.8-19).  The 
owner  must  relate  the  performance  requirements  in  clear  and  unambiguous 
language  (Department  of  the  Air  Force,  2000)  so  that  the  proposers  will  have 
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consistently  similar  understandings  of  the  owner’s  needs  (Beard  et  ah,  2001).  A 
RFP  typically  contains  “design  criteria,  program  requirements,  performance 
specifications,  site  information,  contract  requirements,  selection  procedures,  and 
proposal  requirements  or  deliverables”  (Beard  et  al,  2001;  p.7-8).  Requirements 
and  goals  are  the  focus  of  the  RFP  instead  of  the  detailed  specification  focus  of 
the  Invitation  for  Bid  (IFB)  phase  in  the  traditional  method  (Cushman  &  Loulakis, 
2001). 

4.  Administer  RFP:  The  RFP  is  reproduced  and  issued  for  proposals.  The  RFP 
team  manages  proposers’  inquiries.  Answers  to  inquiries  are  made  through 
amendments  in  the  RFP  and  presented  to  all  package  holders.  The  team  receives 
proposals  and  begins  the  evaluation  process  (Thomburn,  1994).  The  development 
of  proposals  is  extremely  costly  for  design-build  firms.  Proposers  use  extensive 
resources  to  develop  design-build  proposals  (Cushman  &  Loulakis,  2001).  A 
project  that  is  too  small  will  not  offset  the  risk  assumed  by  proposers  to  develop  a 
design  package.  Some  owners  (not  government  agencies)  have  offered  to  pay  a 
stipend  to  proposers  who  were  not  awarded  the  contract  in  order  to  honor  the 
effort  required  to  submit  a  proposal  (Beard  et  al.,  2001).  Offering  such  a  stipend 
will  enhance  relationships  with  design-build  contractors  (Pocock,  2007a)  and  give 
incentive  for  many  proposers,  especially  “A-teams,”  to  compete  (Beard  et  al., 
2001). 

5.  Proposal  Evaluation  and  Award:  The  proposals  received  are  evaluated  according 
to  the  procurement  method  process  and  evaluation  criteria  decided  on  by  RFP 
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development.  The  selection  of  the  One-step  or  Two-step  procurement  method 
determines  the  steps  in  the  process.  The  source  selection  evaluation  board 
(SSEB)  rank  orders  a  range  of  proposals  that  have  met  the  owner’s  requirements. 
The  proposers  in  this  range  are  allowed  to  submit  a  Best  and  Final  Offer  (BAFO) 
which  clarifies  any  questions  and  clarifications  of  the  SSEB  (Thornbum,  1994). 
The  One-step  method  will  then  select  the  proposal  from  this  range  that  gives  the 
best  value  to  the  govermnent  (Webster,  1997).  The  Two-step  method  will  first 
select  the  proposals  that  meet  the  technical  requirements  without  knowing 
proposal  costs.  The  proposals  that  meet  requirements  will  then  be  asked  for  a 
sealed  cost  bid  out  of  which  the  lowest  bid  receives  the  award  (Webster,  1997) 

6.  Design/Construction:  The  final  phase  of  the  process  begins  with  the  notice  to 
proceed  (NTP).  The  NTP  instructs  the  contractor  to  begin  the  design  phase  of  the 
project.  Design  and  cost  reviews  will  be  conducted  as  specified  in  the  RFP.  The 
owner  must  approve  the  construction  documents  before  work  can  begin  on  site 
(Beard  et  al.,  2001).  The  NTP  will  then  be  issued  for  construction  with  the 
Design  Agency  supervising  work  for  quality  control  and  amount  of  completion. 
Work  proceeds  until  the  beneficial  occupancy  date  (BOD)  when  the  government 
accepts  the  facility  (Thornbum,  1994).  The  decision  for  using  fast-track  methods 
must  be  made  in  the  planning  phase  and  not  after  contract  award  (Thornbum, 
1994).  If  fast-track  is  to  be  used,  the  design  will  be  broken  into  two  phases.  The 
first  package  will  contain  site  work,  exterior  utilities,  and  foundations.  The 
second  package  designs  the  rest  of  the  project.  The  constmction  NTP  will  be 
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issued  after  each  design  phase  has  been  approved.  Fast-track  methods  require  a 
larger  staff  for  construction  supervision  and  increases  the  risk  of  errors  and 
problems  due  to  the  fast  pace  of  work  (Thornbum,  1994). 

Project  Delivery  vs.  Procurement 

There  is  a  distinct  difference  between  project  delivery  methods  and  project 
procurement  methods.  Construction  literature  and  previous  studies  have  used  the  terms 
delivery  (Pocock,  1996;  Webster,  1997;  Konchar  &  Sanvido,  1998;  Department  of 
Defense,  2000;  Ling  et  al.,  2004)  and  procurement  (Mouritsen,  1993;  Songer  & 
Molenaar,  1997;  Chan  et  al.,  2002)  synonymously  when  describing  design-build 
methods.  Delivery  and  procurement  are  two  separate  and  important  aspects  of  a  project. 

“A  project  delivery  system  is  the  process  by  which  the  components  of  design  and 
construction-including  professional  services,  labor,  materials,  and  equipment,  as  well  as 
responsibility  for  cost,  schedule,  quality,  and  management-are  combined  under  an 
agreement  that  results  in  a  complete  facility”  (Beard  et  al.,  2001;  p.  1 69).  Examples  of 
delivery  systems  for  construction  are  traditional  (design-bid-build),  design-build 
(classical),  construction  management,  and  partnering  methods.  Project  delivery  methods 
will  be  discussed  later  in  this  chapter. 

“ Procurement  represents  the  purchasing  steps  that  the  owner  or  its  representative 
must  take  to  gain  the  services  and  commodities  required  under  the  chosen  project 
delivery  system”  (Beard  et  al.,  2001;  p.  1 69).  Examples  of  procurement  methods  are  sole 
source,  qualifications-based,  source  selection,  adjusted  low  bid,  and  low  cost  first 
methods  (Beard  et  al.,  2001).  Procurement  methods  range  from  the  relationship  and 
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qualitative  base  of  sole  source  selection  to  the  bottom  line  and  quantitative  driven 
approach  of  low  cost. 

Design-Bid-Build  Procurement  Method 

Low  cost  first  bidding  has  been  the  traditional  way  for  design-bid-build 
construction  procurement  (Figure  4)  (Beard  et  ah,  2001).  Prescriptive  specifications  are 
contained  in  a  contracted  A-E  firm’s  drawings  and  specifications.  An  Invitation  for  Bids 
(IFB)  is  submitted  by  the  owner  and  selects  the  bidder  with  the  lowest  cost.  (Department 
of  the  Air  Force,  2000). 


Procurement  Specification  Solicitation  Solicitation  Award  Contract 

Strategy  Type  Type  Response  Basis  Type 


Figure  4.  Traditional  Procurement  Process  (Mouritsen,  1993;  Figure  2.5) 


Design-Build  Procurement  Methods 

The  three  types  of  design-build  procurement  methods  currently  used  by  the  DOD 
are  the  One-step,  Two-step,  and  Newport  design-build  methods  (Mouritsen,  1993; 
Webster,  1997;  Department  of  the  Air  Force,  2000).  The  One-step  and  Two-step 
methods  are  used  by  the  Army  and  Air  Force  while  the  Newport  design-build  method  is 
used  almost  exclusively  by  the  Navy. 

One-Step  Design-Build 

Also  known  as  the  Source  Selection  method  (Figure  5),  this  method  is  used  by 
federal  agencies  and  is  regulated  by  the  Federal  Acquisition  Regulations  (FAR)  Part  15  to 
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provide  the  “best  value”  to  the  government  (Beard  et  ah,  2001;  p.173).  Technical  and 
price  proposals  are  submitted  for  projects  from  multiple  bidders  after  the  owner  has 
advertised  a  request  for  proposals  (RFP).  Discussions  will  be  held  with  proposers  within 
“the  competitive  range”  for  the  project  (Beard  et  ah,  2001;  p.172).  The  contract  award  is 
selected  from  the  proposers’  best  and  final  offers. 


Procurement  Specification  Solicitation  Solicitation  Award  Contract 

Strategy  Type  Type  Response  Basis  Type 


Figure  5.  One-Step  Design-Build  Procurement  Process  (Mouritsen,  1993;  Figure  2.6) 

Two-Step  Design-Build 

The  Two-step  method  (Figure  6)  takes  advantage  of  technical  proposal  review 
and  low  bidder  award.  Owners  solicit  proposals  using  a  RFP  that  contains  all  project 
details.  The  first  phase  of  the  process  withholds  cost  details  and  identifies  the  three  to 
five  most  qualified  proposers.  The  qualified  proposers  then  submit  sealed  bids  and  the 
project  is  awarded  based  on  the  low  bid  (Department  of  the  Air  Force,  2000). 


Procurement  Specification  Solicitation  Solicitation  Award  Contract 

Strategy  Type  Type  Response  Basis  Type 


Figure  6.  Two-Step  Design-Build  Procurement  Process 
(Mouritsen,  1993;  Figure  2.7) 
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Newport  Design-Build 

The  Newport  design-build  method  removes  the  costly  initial  technical  proposals 
required  for  RFP  preparation  (Figure  7).  This  method  “combines  the  single  source  of 
responsibility  concept  with  lump  sum  competitive  bidding,  awarding  the  contract  to  the 
lowest  bidder”  (Mouritsen,  1993;  p.  47). 


Procurement  Specification  Solicitation  Solicitation  Award  Contract 

Strategy  Type  Type  Response  Basis  Type 


Figure  7.  Newport  Design-Build  Procurement  Process 
(Mouritsen,  1993;  Figure  2.8) 


An  in-depth  analysis  of  design-build  procurement  methods  for  MILCON  is 
beyond  the  scope  of  this  research.  However,  design-build  procurement  was  analyzed  in 
the  previously  mentioned  study  by  El  Wardani  et  al.  (2006).  The  study  built  off  the 
Sanvido  and  Konchar  (1997)  study  that  will  be  discussed  later.  El  Wardani  et  al. 
collected  data  from  76  of  the  155  Sanvido  and  Konchar  design-build  projects.  Sole 
source,  qualifications-based,  best  value,  and  low  bid  procurement  methods  were  analyzed 
against  the  performance  metrics  in  Figure  8  and  facility  type.  Significant  results  of  the 
study  were:  1)  the  highest  cost  growth  resulted  from  projects  using  the  low  bid 
procurement  method  and  2)  the  qualifications-based  method  resulted  in  the  lowest  cost 
growth.  However,  due  to  a  small  sample  size,  the  study  was  unable  to  significantly 
determine  the  one  procurement  method  best  suited  for  design-build. 
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Project  Selection  Criteria 

Professionals  disagree  on  when  design-build  should  be  used  as  a  project  delivery 
system  (Cushman  &  Loulakis,  2001).  Early  design-build  studies  reported  that  the  design- 
build  delivery  method  should  only  be  used  for  simple,  repetitive  (such  as  housing)  or 
non-technical  projects  (Buckingham,  1989).  The  unfamiliarity  with  the  design-build 
process  kept  many  project  managers  skeptical  of  the  broad  applicability  of  the  new 
approach  (Buckingham,  1989;  Mouritsen,  1993;  Molenaar  &  Songer,  1998).  The 
pendulum  has  now  shifted  away  from  the  early  thoughts  of  limited  use  for  design-build 
(Buckingham,  1989;  Cushman  &  Loulakis,  2001).  However,  some  feel  that  design-build 
can  and  should  be  used  for  any  and  all  projects  (Pocock,  1996;  Webster,  1997;  Langley, 
2007b),  while  others  have  a  more  conservative  viewpoint  of  using  design-build  for  the 
right  application  (Songer  &  Molenaar,  1997;  Konchar  &  Sanvido,  1998;  Department  of 
the  Air  Force,  2000;  Ling  et  al.,  2004).  Additionally,  owners  and  agents  must  realize  that 
the  choice  of  delivery  methods  is  only  one  piece  of  the  puzzle.  Project  success  also 
depends  on  the  correct  consideration  of  procurement  method,  risk  allocation,  and  owner’s 
financing  abilities  (Cushman  &  Loulakis,  2001). 

When  to  use  design-build 

Public  projects  have  traditionally  been  selected  for  delivery  via  design-build  on  a 
subjective  basis  (Molenaar  &  Songer,  1998).  Recently,  key  project  characteristics  have 
been  identified  and  incorporated  into  models  for  delivery  method  selection.  Additionally, 
facility  type  has  emerged  as  a  factor  that  influences  the  decision  to  use  design-build. 
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Project  characteristics  for  successful  design-build  have  been  extensively  studied, 
thereby  providing  managers  with  multiple  resources  for  delivery  method  decision 
making.  Academic  studies  have  identified  the  key  characteristics  for  design-build 
consideration  (Songer  &  Molenaar,  1997),  produced  models  for  public  sector  design- 
build  selection  (Molenaar  &  Songer,  1998),  and  predicting  performance  of  design-build 
and  traditional  projects  (Ling  et  ah,  2004). 

Identification  of  significant  project  characteristics  enabled  the  construction  of  a 
project  selection  model.  Public  owners  and  project  managers  should  consider  using  the 
Design-Build  Selector  (DBS)  (http://www.colorado.edu/engineering/civil/db/)  to  rate  a 
project’s  overall  appropriateness  for  using  the  design-build  method  (Molenaar  &  Songer, 
1998).  However,  to  fully  reap  the  benefits  of  the  DBS  selection  tool,  owners  need  to 
understand  the  significant  project  characteristics  used  to  develop  the  tool.  Characteristics 
that  significantly  predict  project  success  are  grouped  into  four  categories:  project,  owner, 
market,  and  relationship  characteristics  (Molenaar  &  Songer,  1998). 

Project  Characteristics 

The  project  characteristics  that  predict  project  success  are  scope  definition, 
schedule  definition,  budget  definition,  and  project  complexity. 

•  Scope  definition:  A  well  defined  scope  and  a  shared  understanding  of  scope  are 
the  two  characteristics  that  have  the  most  impact  on  project  success  (Songer  & 
Molenaar,  1997).  Success  results  from  a  clear  RFP  that  conveys  the  owner’s 
requirements  and  goals  but  is  not  constrained  by  initial  design  so  as  to  leave  room 
for  contractor  input  (Molenaar  &  Songer,  1998). 
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•  Schedule  definition:  Projects  with  established  completion  dates  (Songer  & 
Molenaar,  1997)  and  driven  by  schedule  are  appropriate  for  design-build.  Such 
projects  require  owners  to  load  their  involvement  in  design  on  the  front  end  of  the 
project  (Molenaar  &  Songer,  1998). 

•  Budget  definition:  An  established  budget  (Songer  &  Molenaar,  1997)  and  the 
conveyance  of  budget  through  a  constraint  on  scope  will  enhance  project  success 
(Molenaar  &  Songer,  1998). 

•  Project  complexity:  Technologically  advanced  and  complex  projects  using 
design-build  delivered  less  administrative  burden  and  higher  overall  satisfaction 
(Songer  &  Molenaar,  1997;  Molenaar  &  Songer,  1998).  However,  the  owner 
must  understand  the  project  requirements  in  order  to  effectively  communicate 
project  complexity  to  proposers  (Beard  et  ah,  2001).  Against  earlier  thought, 
simple  and  repetitive  projects  were  not  correlated  with  design-build  success 
(Songer  &  Molenaar,  1997). 

•  Size  of  project:  Project  size,  indicated  by  cost,  must  be  sufficient  enough  to  offset 
the  proposer’s  expense  to  submit  a  proposal  (Songer  &  Molenaar,  1997;  Cushman 
&  Loulakis,  2001;  Beard  et  ah,  2001;  Pocock,  2007a). 

Owner  Characteristics 

The  owner  characteristics  that  predict  project  success  are  owner/agency 
experience  and  owner/agency  staffing. 

•  Owner/Agency  experience:  The  owner’s  construction  sophistication  must  be 
adequate  to  precisely  define  the  scope  (Songer  &  Molenaar,  1997).  Perfonnance 
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increases  as  the  owner  gains  experience.  A  typical  project  should  be  used  for  the 
owner’s  first  design-build  project  (Molenaar  &  Songer,  1998).  Inexperienced 
government  agencies  are  advised  against  using  design-build  for  the  first  time 
without  experienced  consultants  (Department  of  the  Air  Force,  2000). 

•  Owner/Agency  staffing:  An  allure  of  design-build  is  the  ability  to  do  more  with 
less  (Molenaar  &  Songer,  1998).  However,  owners  must  be  able  to  dedicate 
adequate  staff  to  the  specific  design-build  project  (Songer  &  Molenaar,  1997)  to 
meet  the  demands  of  each  project.  Some  projects  (fast-track,  schedule-driven) 
require  additional  staff  to  develop  the  RD  and  RFP,  as  well  as  meet  the  demands 
of  the  expedited  schedule  (Thornbum,  1994). 

Market  characteristics 

The  market  characteristic  that  predict  project  success  is  design-builder  experience. 

•  Design-builder  experience:  The  availability  of  experienced  design-builders  can 
be  an  issue  that  impacts  success  (Songer  &  Molenaar,  1997;  Department  of  the 
Air  Force,  2000).  Experienced  design-builders  enhance  constructability  and 
innovation  leading  to  cost  and  schedule  savings.  Experienced  contractors  perform 
well  with  performance-based  RFP  processes  and  specifications  (Molenaar  & 
Songer,  1998). 

Relationship  Characteristics 

The  relationship  characteristics  that  predict  project  success  are  design-builder 
selection  and  design-builder  prequalification. 
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•  Design-builder  selection:  Selection  of  design-builders  through  a  combination  of 
price  and  quality  requirements  best  meets  owner’s  expectations  and  influences 
administrative  burden.  Although  difficult  for  the  public  sector,  developing  a  short 
list  of  proven  qualified  bidders  to  choose  from  enhances  performance  (Molenaar 
&  Songer,  1998). 

•  Design-builder  prequalification:  Prequalification,  through  past  performance, 
keeps  the  best  proposers  in  the  field  and  reduces  frivolous  proposals.  A  limited 
number  of  quality  proposals  encourage  competition  (Molenaar  &  Songer,  1998). 
Ling  et  al.  (2004)  also  developed  a  comprehensive  list  of  project  characteristics 

that  impact  cost,  quality,  and  schedule  perfonnance  metrics.  The  59  project,  owner, 
consultant,  and  contractor  characteristics  were  used  to  develop  a  model  to  predict  design- 
build  and  design-bid-build  performance  across  the  performance  metrics  identified  in 
Figure  8.  The  study  produced  two  robust  design-build  and  design-bid-build  prediction 
models  for  time  performance  and  two  design-build  quality  prediction  models  (Ling  et  al., 
2004). 

Facility  Type 

Understanding  and  identifying  the  characteristics  of  the  project  and  the  attributes 
of  the  management  team  in  order  to  detennine  what  product  delivery  method  to  use  can 
be  complicated  and  time  consuming.  The  type  of  facility  an  owner  is  trying  to  build  is 
much  easier  to  understand  than  the  characteristics  of  the  projects.  The  type  of  facility 
that  lends  itself  to  the  design-build  method  has  become  of  interest  to  the  design-build 
community. 
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Two  studies  classified  facilities  as  light  industrial,  multi-story  dwelling,  simple 
office,  complex  office,  heavy  manufacturing,  and  high  technology  to  detennine  what 
delivery  method  (Konchar  &  Sanvido,  1998)  and  procurement  method  (El  Wardani, 
2006)  are  most  effective.  Significant  results  were  obtained  by  Konchar  &  Sanvido 
(1998)  when  delivery  method  depending  on  facility  type  was  analyzed  for  the 
performance  metrics  displayed  in  Figure  8.  Design-build  significantly  out-performed 
design-bid-build  in  light  industrial  (unit  cost,  construction  speed,  delivery  speed,  system 
quality),  multi-story  dwelling  (intensity),  simple  office  (intensity),  complex  office 
(intensity,  turnover  quality),  and  high  technology  (cost  growth,  turnover  quality,  system 
quality)  facility  types  (El  Wardani,  2006).  However,  due  to  small  sample  size,  the  type 
of  procurement  method  did  not  yield  significant  cost,  time,  or  quality  results  depending 
on  facility  type  (El  Wardani,  2006). 

The  Blue  Book  (Department  of  the  Air  Force,  2000)  directs  AF  project  managers 
to  consider  facilities  that  use  private  sector  standards,  commercial  standards,  and  AF 
design  guides  as  appropriate  MIFCON  projects  for  use  with  the  design-build  method 
(Table  8).  Standard  design  specifications,  similar  to  existing  projects,  have  been 
identified  as  a  characteristic  for  successful  design-build  (Songer  &  Molenaar,  1997). 
However,  the  rationale  for  selecting  design-build  for  these  facility  types  is  that  extensive 
guidance  and  standards  exist  that  will  aid  project  development  when  formal  contract 
documents  are  not  created.  Although  the  AF  recommends  certain  facilities  for  design- 
build,  the  most  effective  delivery  method  depending  on  facility  type  for  AF  MIFCON 
projects  has  not  been  quantitatively  evaluated. 
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Table  8.  Facility  Standards 
(Department  of  the  Air  Force,  2000;  Table  8-1) 


Facility  Type 

Private  Sector 
Standards 

Commercial 

Standards 

AF  Design  Guides 

Housing 

X 

X 

X 

Hangars 

X 

X 

Admin/HQ 

X 

X 

Medical  Clinics 

X 

Maintenance  Fac 

X 

X 

Dorms/TLFs 

X 

Child  Dev  Centers 

X 

X 

X 

Training  Facilities 

X 

Dining  Facilities 

X 

X 

Warehouse 

X 

X 

MWR  Facilities 

X 

X 

Flightline  Facilities 

X 

Project  Success  Criteria 

A  thorough  comparison  between  design-build  and  traditional  delivery  methods 
requires  the  identification  of  project  success  criteria  that  will  be  measured.  This  section 
reviewed  the  available  literature  on  project  success  as  it  pertains  to  design-build 
construction.  Additionally,  the  metrics  used  to  measure  successful  MILCON  projects 
were  identified.  Understanding  the  metrics  used  by  the  industry  to  measure  project 
success  will  develop  the  metrics  to  be  used  by  this  study  to  compare  delivery  methods. 
Project  Success 

Chan  et  al.  defined  project  success  as:  “the  degree  to  which  project  goals  and 
expectations  are  met”  (2002;  p.  12 1).  Project  performance  indicators  define  the  criteria 
used  to  measure  the  success  of  the  project.  Project  performance  has  historically  been 
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measured  using  three  key  criteria:  cost,  time,  and  performance  or  quality  (Meredith  & 
Mantel,  2006;  Chan  et  ah;  2002;  Beard  et  ah;  2001;  Konchar  &  Sanvido,  1998;  Songer  et 
ah,  1997).  The  end  product  consists  of  goals  for  each  criteria  making  up  a  target  for  the 
entire  project. 

The  generalized  criteria  of  cost,  time,  and  perfonnance  encompass  many  factors 
that  define  success  differently  for  each  project  participant.  Clients,  owners,  designers, 
and  builders  each  have  differing  project  success  criteria  (Mouritsen,  1993;  Chan  et  ah, 
2002).  Project  success  criteria  also  change  as  the  project  progresses  through  the  three 
phases  of  construction:  preconstruction,  construction,  and  post  construction  phases  (Chan 
et  ah,  2002). 

Project  Performance  Measures 

Chan  et  al.  (2002)  conducted  a  comprehensive  study  of  construction  literature  in 
order  to  develop  success  criteria  for  the  design-build  delivery  method.  The  results  of  the 
study  grouped  project  performance  measures  into  objective  and  subjective  categories. 
Objective  measures  include  time,  cost,  health  and  safety,  and  profitability.  Subjective 
measures  include  quality,  technical  performance,  functionality,  productivity,  satisfaction, 
and  environmental  sustainability.  The  framework  developed  by  Chan  et  al.  captured  the 
success  criteria  and  performance  metrics  used  by  researchers  to  measure  design-build 
success. 

Industry  Performance  Metrics 

Objective  and  subjective  performance  measures  are  operationalized  through 
success  criteria  categories  (Songer  &  Molenaar,  1997)  which  are  in  turn  broken  down 
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into  performance  metrics.  Figure  8  summarizes  the  performance  metrics  used  by  the 
premier  design-build  studies  to  measure  and  compare  the  effectiveness  of  the  design- 
build  and  traditional  delivery  methods. 


Performance  Measures  (5) 

Objective  Measures 

Cost 

Time 

Health  and  Safety 

Profitability 

Success  Criteria  (2) 


On  Budget 


Performance  Metric 


Unit  Cost 

4,  6,  8 

%  Cost  Growth 

1,  3,  4,  6,  7,  8 

CWE/PA  ratio 

7 

%  Schedule  Growth 

1,  3,  4,  6,  7,  8 

Construction  Speed 

4,  6,  8 

Delivery  Speed 

4,  6,  8 

Intensity 

4,  6,  8 

Design  Authorization 

7 

Design  Completion 

7 

President's  Budget  Projects 

7 

Consgressional  Insert  Projects 

7 

Construction  Timeline 

7 

t> 


|Timely  ( 


Quality 


Technical  Performance 


Functionality 


Productivity 


Satisfaction 


Environmental  Sustainability] 


Key: 

1 

2 

3 

4 


| Owner's  Administrative  Burden  |6 


J  ^  [Minimizes  Construction  Aggravation  :=>[ 


Owner's  Satisfaction 


Pocock  (1996)  5 

Songer  &  Molenaar  (1997)  6 

Webster  (1997)  7 

Konchar  &  Sanvido  (1998)  8 


Chan  et  al  (2002) 

Ling  et  al.  (2004) 

Fox  (2006) 

El  Wardani  etal.(2006) 


Figure  8.  Project  Performance  Measures 
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Dirtkicker  Criteria 


The  Air  Force  Civil  Engineer  created  Dirtkicker  criteria  to  measure  the  successful 
execution  and  management  of  AF  MILCON  by  the  MAJCOMs.  Project  managers, 
MAJCOMs,  and  the  Anny  Corps  of  Engineers  use  the  Dirtkicker  metrics  to  assess  the 
progress  and  perfonnance  of  AF  MILCON  projects.  The  best  performing  large  and  small 
MAJCOM  would  be  awarded  additional  Operations  and  Maintenance  (O&M)  funds. 
Dirtkicker  criteria  provided  metrics  for  project  design,  award,  construction,  and  financial 
closure  (Robbins,  2003).  Table  9  outlines  the  goals  set  forward  by  the  FY06  Dirtkicker 
Award  criteria. 


Table  9.  FY06  Dirtkicker  Award  Criteria  (Fox,  2006) 


Criteria 

Goal 

Design 

Use  of  Old  P&D  Funds 

All  FY04  and  prior  year  funds  obligated.  FY05  funds  at  least  80% 
obligated.  Criteria  are  measured  for  the  entire  MAJCOM’s  program 

Design  Authorization 

MAJCOMs  Issue  Field  Design  Instructions  to  agents  within  30  days  after 
ILEC  issues  Project  Design  Authorization 

Design  Completion 

FY07  President’s  Budget  projects  design  complete  (i.e.,  100%)  NLT  30  Sep 
06 

Award 

President's  Budget 
Projects 

FY06  President’s  Budget  projects  awarded  NLT  31  Mar  06. 

Congressional  Insert 
Projects 

FY06  Congressional  Insert  projects  awarded  NLT  30  Sep  06 

CWE/PA  Ratio 

The  sum  of  the  Award  CWE  (or  Design  CWE  for  unawarded  projects) 
divided  by  the  sum  of  the  original  PA  (e.g.,  before  any  rescission,  if 
applicable)  for  all  FY06  projects,  grouped  by  MAJCOM  is  less  than  or 
equal  to  97%  (.97). 

Construction 

Cost  Growth 

5%  Cost  Growth  or  less,  which  is  when  the  ratio  of  current  contract  cost 
to  original  contract  cost  is  1.05  or  less. 

Schedule  Growth 

10%  Schedule  Growth  or  less,  which  is  when  the  performance  days  (i.e., 
NTP  to  BOD)  divided  by  the  original  estimated  performance  days  is  1.1  or 
less 

Construction  Timeline 

Construction  Timeline  is  the  frequency  with  which  a  MAJCOM 
constmcts  its  projects  at  or  below  specified  ‘Target  Days  ’,  which  are 
determined  by  Programmed  Amounts.  (MILCON  overview  Table  3) 

Financial 

Closure 

Timely  Closeout 

Financial  closeout  within  12  Months  (15  months  forOCONUS). 
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Dirtkicker  criteria  provide  the  standards  by  which  Army  and  AF  leadership 
determines  MILCON  project  success.  Military  leadership  uses  the  project  data  in 
ACES-PM  to  calculate  the  Dirtkicker  values.  Therefore,  these  metrics  provide  a  common 
basis  to  communicate  the  results  of  this  study  to  AF  program  managers. 

Previous  Studies 

The  purpose  of  this  section  is  to  consolidate  previous  investigations  into  similar 
questions  and  suggest  where  this  proposed  inquiry  may  differ  in  tenns  of  approach,  data 
source,  and  subject  matter  focus.  Table  10  displays  the  attributes  of  the  previous  studies 
researched  that  have  impacted  the  direction  of  this  thesis. 

Buckingham,  1989 

Buckingham  (1989)  was  one  of  the  first  studies  conducted  by  the  AF  in  analyzing 
the  possible  use  of  design-build  as  a  solution  to  problems  with  traditional  MILCON 
construction.  The  AF  was  new  to  design-build  with  only  four  projects  completed  at  the 
time  (one  FY84,  one  FY85,  two  FY87).  Private  sector  companies  were  interviewed  for 
their  opinions  on  the  design-build  method.  The  study  then  conducted  a  case  study  on  the 
FY85  AF  Communications  Command  HQ  Facility  at  Scott  AFB,  IL  and  the  FY87  Base 
Dining  Facility  at  Robbins  AFB,  GA.  The  study  suggested  only  small,  simple  MILCON 
projects  should  use  design-build.  An  insufficient  track  record  existed  and  AF  personnel 
were  too  inexperienced  to  detennine  if  design-build  was  more  effective  (Buckingham, 
1989). 
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Table  10.  Comparison  of  Previous  Studies 
(adapted  from  Konchar  &  Sanvido,  1998) 


'  Project  Delivery  Study  j 

Attribute 

Konchar  & 

El  Wardani 

Buckingham 

Mouritsen 

Pocock 

Webster 

Sanvido 

Allen 

Hale 

et  al. 

(1989) 

(1993) 

(1996) 

(1997) 

(1998) 

(2001) 

(2005) 

(2006) 

!  (a)  Data  collecion  Instment  j 

Objective  data 

- 

X 

X 

X 

X 

X 

X 

X 

Owner  quality  data 

- 

- 

- 

- 

X 

X 

- 

X 

j  (b)  Systems  Compared  j 

Design-bid-build 

- 

X 

X 

X 

X 

X 

X 

- 

Design-build 

X 

X 

X 

X 

X 

X 

X 

X 

Construction  management 

- 

- 

- 

- 

X 

- 

- 

- 

Partnering 

- 

- 

X 

- 

- 

- 

- 

- 

Combination 

- 

- 

X 

X 

- 

- 

- 

- 

]  (c)  Type  of  comparison  | 

Univariate  analysis 

- 

X 

X 

X 

X 

Multivariate  analysis 

- 

- 

X 

- 

X 

Number  of  variables 

NA 

5 

13 

5 

13 

!  (d)  Survey  research  j 

Project/respondend  specific 
data  collection 

X 

X 

X 

X 

X 

X 

X 

X 

Nonresponse  study 

- 

- 

- 

- 

X 

- 

- 

- 

Case  Study 

X 

X 

- 

- 

- 

- 

- 

- 

Opinion  poll 

- 

- 

- 

- 

- 

X 

- 

- 

Empirical  measures 

- 

X 

X 

X 

X 

X 

X 

X 

Private  sector 

X 

- 

- 

X 

- 

- 

X 

Public  sector 

X 

X 

X 

X 

X 

X 

X 

X 

MILCON 

X 

X 

X 

X 

- 

X 

X 

- 

Facility  classification 

- 

X 

- 

- 

X 

X 

X 

X 

Number  of  projects 

2 

11 

209 

29* 

351 

110 

77 

76 

Number  of  subjects 

4 

NA 

NA 

NA 

NA 

NA 

NA 

NA 

|  NA=  not  applicable 

*  Plus  146  Pocock  projects 

Mouritsen,  1993 

One  of  the  first  quantitative  MILCON  studies  of  design-build  was  accomplished 
by  Mouritsen  in  1993  for  the  US  Navy.  The  study  analyzed  1 1  child  care  centers 
constructed  in  FY90  (five  traditional,  two  One-step  design-build,  four  Newport  design- 
build).  Results  showed  a  design-build  cost  savings  range  of  15.5%  for  the  one-step 
method  to  21.9%  using  the  Newport  design-build  method,  but  without  a  statistical 
analysis  completed.  Project  delivery  time  was  claimed  to  be  cut  in  half,  but  without 
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adequate  methodological  support.  The  observations  of  the  study  determined  that  the 
design-build  method  should  be  used  for  all  projects.  The  study  was  limited  by  small 
sample  size,  lack  of  project  data  in  the  information  system,  and  the  Navy’s  inexperience 
with  the  process.  As  with  the  Buckingham  (1989)  study,  the  study  analyzed  the  few 
existing  pilot  projects  so  a  sufficient  track  record  did  not  exist  (Mouritsen,  1993). 

Pocock,  1996 

The  research  of  Pocock  (1996)  was  the  first  quantitative  delivery  method 
MILCON  study  found  by  this  research  to  use  statistical  significance  to  validate  findings. 
Pocock  compared  partnered,  design-build,  and  combination  alternative  methods  to 
design-bid-build  projects  using  the  performance  indicators  in  Figure  8.  Pocock  gathered 
209  (90  traditional,  63  partnering,  40  design-build,  16  combination)  projects  from  FY88 
to  FY95  from  the  US  ACE  Automated  Management  and  Progress  Reporting  System 
(AMPRS).  Combination  projects  used  characteristics  from  traditional,  design-build,  and 
constructability  delivery  approaches.  The  study  also  defined  and  measured  the  degree  of 
interaction  (DOI),  which  is  the  amount  of  interaction  between  design  and  construction. 
Pocock  developed  a  method  to  calculate  the  DOI  for  each  project  in  order  to  compare  the 
interaction  levels  for  the  different  delivery  methods  (Pocock,  1996). 

The  results  of  the  study  showed  that  design-build  was  the  best  category  for 
significantly  less  cost  growth  and  design  deficiencies  of  all  delivery  methods.  The  overall 
conclusion  was  that  alternative  approaches  have  a  significantly  better  performance  and 
degree  of  interaction  than  traditional  projects  (Pocock,  1996).  Table  1 1  summarizes  the 
results  of  the  Pocock  (1996)  study  along  with  the  Webster  (1997)  study. 
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Webster,  1997 

Webster’s  study  was  a  continuation  of  Pocock’s  1996  dissertation  thesis.  The 
research  sought  to  verify  Pocock’s  methods,  detennine  if  design-build  improved  over 
time,  and  investigate  subjective  success  criteria.  Webster  compared  29  additional  design- 
build  FY91  to  FY96  MILCON  projects  with  Pocock’s  traditional,  combination,  and 
design-bid  projects  over  the  same  four  performance  indicators.  Interviews  were 
conducted  to  gather  the  data  for  the  subjective  indicators  of  user  satisfaction,  project 
management  satisfaction,  and  experience  with  the  design  build  process. 

Results  showed  (Table  1 1)  design-build  data  was  significantly  (p<0.05)  better 
than  traditional  projects  for  schedule  growth,  modifications  per  million  dollars, 
modifications  due  to  design  deficiencies.  However,  only  schedule  growth  showed  a 
significant  improvement  over  time  for  design-build  projects.  Additionally,  the  design- 
build  projects  failed  to  be  statistically  better  than  the  combination  projects  for  all 
categories.  The  subjective  analysis  of  satisfaction  only  resulted  in  a  brief  discussion  of 
two  design-build  projects.  Therefore,  the  subjective  analysis  did  not  provide  enough  of  a 
response  for  generalization  for  all  design-build  projects  (Webster,  1997). 

This  study  succeeded  in  validating  the  statistical  analysis  of  MILCON  projects 
proposed  by  Pocock  (1996).  However,  the  results  of  the  study  suffered  due  to  a  small 
sample  size  of  new  design-build  projects  and  how  recently  it  was  conducted  after  the 
Pocock  study.  The  MILCON  process  is  slow  to  change,  so  it  is  not  reasonable  to  expect 
much  of  a  change  in  only  one  year. 
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Table  11:  Average  Project  Performance  (Webster,  1997;  Table  4.1) 


Performance  Indicators 

Study 

Design-build 

Traditional 

Pocock  study 
Design-build 

Combination 

Cost  Growth  (%) 

6.5 

8  48 

6  37 

10  44 

Variance 

113 

141 

59 

100 

p(T,-t),one  tail 

0  20294 

- 

0  1135 

02460 

15  04 

27  76 

26  23 

18  76 

Variance 

906 

1099 

1285 

541 

p(T,  -t),one  tail 

0  03982 

- 

0  4107 

0  0978 

Modifications  per  Million  Dollars 

5.43 

8.3 

68 

4  95 

Variance 

10 

37 

36 

7 

00007 

- 

0  1081 

0  0004 

Modifications  due  to  Design  Deficiencies  (%) 

24.4 

41  84 

9  39 

15  18 

Variance 

459 

344 

157 

329 

p(T.=t),one  tail 

2.16E-04 

- 

6.93E-15 

1  34E-OS 

Konchar  &  Sanvido  (1998) 

This  work  is  considered  as  the  industry  benchmark  and  is  cited  in  the  leading 
guides  (Cushman  &  Loulakis,  2001;  Loulakis,  2003),  textbooks  (Beard  et  ah,  2001;  Link, 
2006),  and  studies  (Ling  et  ah,  2004;  El  Wardani  et  ah,  2006).  Sponsored  by  the 
Construction  Industry  Institute  (CII),  the  study  analyzed  351  (traditional,  design-build, 
construction  management  (CM))  private  and  public  projects.  The  study  compared  the 
delivery  methods  against  the  performance  metrics  in  Figure  8.  Additionally,  the 
performance  of  the  delivery  methods  was  compared  over  six  facility  types:  light 
industrial,  multi-story  dwelling,  simple  office,  complex  office,  heavy  manufacturing,  and 
high  technology.  Results  showed  that  design-build  was  significantly  better  than  CM  at 
risk  and  traditional  methods  over  cost,  schedule,  and  quality  categories.  The  study  also 
identified  which  type  of  facility  was  best  suited  for  each  delivery  method.  El  Wardani  et 
al.  (2006),  as  previously  discussed,  built  off  this  study  to  analyze  the  effects  of 
procurement  types  for  design-build  projects. 
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Allen,  2001 

This  study  compared  36  design-build  and  74  design-bid-build  Navy  Bachelor 
Enlisted  Quarters  (BEQ)  MILCON  projects  from  FY1996  to  FY2000.  Projects  were 
further  categorized  as  horizontal  or  vertical  projects.  The  data  was  retrieved  from  the 
Navy  Financial  Infonnation  System  (FIS).  The  results  showed  that  design-build  projects 
have  less  award  growth,  cost  growth,  and  schedule  growth  with  better  construction 
placement  than  design-bid-build.  The  qualitative  portion  of  this  study  revealed  that  the 
design-bid-build  method  outperfonned  design-build  in  turnover  process  quality  and 
system  performance  quality.  While  the  study  compared  performance  metric  means,  the 
analysis  lacked  statistical  analysis  to  determine  significant  difference  levels.  The  study 
thoroughly  analyzed  one  facility  type  with  a  larger  sample  size  than  previous  studies. 
However,  the  study  analyzed  the  first  projects  using  design-build  by  the  Southwest 
Division  of  the  NAVFAC.  Therefore,  the  project  managers  were  new  to  the  design-build 
process  and  it  would  be  beneficial  to  analyze  projects  after  a  sufficient  design-build  track 
record  has  been  established. 

Hale,  2005 

This  more  recent,  yet  unrelated  to  Allen  (2001),  study  also  investigated  Navy 
BEQs  with  a  smaller  sample  size  of  38  design-build  and  39  design-bid-build  projects 
from  FY  1995  to  FY  2004.  Project  data  was  gathered  from  the  FIS.  A  statistical  analysis 
of  variance  (ANOVA)  was  used  to  determine  if  the  comparison  of  the  performance 
metrics  was  significant.  Design-build  showed  to  be  significantly  better  for  the  metrics  of 
total  project  duration  (date  of  contract  award  to  completion),  project  duration  per  bed, 
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time  growth,  and  cost  growth.  The  cost  per  bed  metric  favored  design-build,  but  not  at  a 
significant  level.  As  with  Allen  (2001),  one  shortfall  of  this  study  was  that  it  only 
analyzed  one  homogenous  facility  type  with  recommendations  for  the  use  of  design-build 
for  other  facility  types.  Another  shortcoming  of  this  study  was  limiting  the  total  project 
time  metric  to  the  contract  period.  The  total  project  time  for  MILCON  projects  should 
look  at  the  entire  process  including  the  planning,  programming,  and  design  portions  of 
the  projects  and  not  just  the  construction  portion. 

Air  Force  Studies,  2006 

The  performance  of  design-build  verses  traditional  delivery  methods  became  of 
great  interest  among  the  AF  civil  engineer  community  in  2006.  A  report  from  Air  Staff 
in  the  Weekly  Activity  Report  (WAR)  showed  that  traditional  projects  had  a  higher 
probability  of  cost  growth  exceeding  Dirtkicker  goals  and  design-build  was  increasingly 
staying  within  or  below  cost  (WAR,  2006).  The  WAR  report  contradicted  an 
unpublished  study  by  AF  project  managers  at  the  Air  Combat  Command  (ACC).  The 
ACC  study  used  Dirtkicker  criteria  performance  metrics  to  compare  the  delivery  method 
with  facility  type.  The  study  detennined  that  the  traditional  method  outperformed  the 
design-build  method  in  every  facility  type  category  except  child  development  centers  and 
fire/crash  facilities  (Hunt,  2006).  The  confusion  on  the  perfonnance  of  MILCON  design- 
build  projects  is  represented  by  these  studies,  MAJCOM  opinion,  and  anecdotal  answers 
resulted  in  the  call  for  a  quantitative  study  of  MILCON  delivery  methods  (Glardon, 

2006). 
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Summary 

The  design-build  delivery  method  is  no  longer  a  new  concept  for  MILCON 
execution.  However,  a  consensus  has  not  been  reached  by  AF  project  managers  as  to  the 
correct  time,  place,  and  application  for  using  design-build.  The  private  sector  has 
embraced  the  design-build  method  based  on  academic  studies  (Beard  et  ah,  2001).  While 
the  number  of  academic  studies  investigating  design-build  has  increased  (Pietroforte, 
2004),  few  have  focused  on  delivery  methods  for  MILCON.  The  investigation  of  design- 
build  literature  identified  the  following  gaps: 

•  All  but  one  of  the  MILCON  studies  reviewed  were  conducted  when  a  proven 
track  record  of  projects  did  not  exist  and  before  the  approval  of  design-build  for 
widespread  use. 

•  An  in-depth  study  of  Air  Force  MILCON  delivery  methods  has  not  been 
conducted  since  1989. 

•  The  most  recent  MILCON  studies  only  focused  on  Navy  Bachelors  Enlisted 
Quarters  without  addressing  other  facility  types. 

•  MILCON  design-build  study  sample  size  was  limited  due  to  few  available 
projects. 

•  The  inexperience  of  AF  project  managers  with  the  design-build  method  hindered 
a  qualitative  study  of  delivery  method  as  a  predictor  of  project  success. 

•  The  performance  of  the  design-build  method  over  time  has  not  been  sufficiently 
analyzed  in  order  to  detennine  if  the  method  has  improved. 
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•  The  type  of  facility  for  using  design-build  has  been  recommended  in  the  Blue 
Book,  but  an  analysis  of  facility  type  for  MILCON  design-build  has  not  been 
conducted. 

•  Dirtkicker  criteria  are  used  as  the  measure  of  success  for  MILCON  projects.  No 
formal  study  has  used  Dirtkicker  criteria  to  compare  design-build  and  traditional 
methods. 

•  An  in-depth  study  has  not  been  conducted  to  compare  total  project  time  for 
MILCON  from  the  start  of  initial  design  to  completion  of  construction. 

This  research  will  attempt  to  address  the  gaps  identified  in  the  previous  research. 
The  design-build  and  traditional  design-bid-build  methods  will  be  compared  for  the 
MILCON  application.  The  literature  review  identified  how  project  success  is  measured. 
Eight  performance  metrics  will  be  used  by  this  study  and  include:  unit  cost,  cost  growth, 
schedule  growth,  construction  speed,  modifications  per  million  dollars,  CWE/PA  ratio, 
construction  timeline,  and  total  project  time.  A  complete  description  of  the  calculations 
used  for  the  metrics  discussed  above  and  the  methods  used  for  this  research  will  be 
included  in  Chapter  3,  Methodology. 
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III.  Methodology 


Chapter  Overview 

This  chapter  describes  the  methodology  used  to  compare  the  design-build  delivery 
method  to  the  traditional  design-bid-build  method  for  Air  Force  (AF)  military 
construction  (MILCON)  projects.  The  procedures  used  by  this  study  are  organized  into 
three  sections:  data  source,  data  collection,  and  data  analysis.  Each  section  will  explain 
the  definitions,  decisions,  and  criteria  used  for  the  study  data  analysis. 

Data  Source 

The  first  step  of  data  analysis  was  to  identify  a  data  source  that  contained 
consistent  and  accurate  project  information  for  traditional  and  design-build  projects. 
Military  construction  projects  provide  researchers  with  consistent  and  accurate  data  due 
to  Congressional  oversight  and  established  contracting  and  management  procedures 
(Pocock,  1996).  MILCON  project  information  is  documented  by  design  agents  in  three 
main  databases.  Projects  managed  by  the  Army  Corps  of  Engineers  will  be  entered  into 
the  Automated  Review  and  Management  System  (ARMS).  Projects  which  use  the 
NAVFAC  as  the  design  agent  are  tracked  in  the  NAVFAC  Construction  Management 
Information  System  (CMIS).  Regardless  of  design  agent,  every  AF  MILCON  project  is 
documented  and  managed  in  the  Automated  Civil  Engineer  System  -  Project 
Management  (ACES-PM)  database  from  the  planning  and  programming  stage  through 
construction. 
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ACES  tracks  an  enormous  amount  of  project  information  in  order  to  manage  real 
property,  housing,  fire  department,  government  provided  furnishings,  facility 
maintenance  and  scheduling,  personnel  and  training,  military  construction,  environmental 
impact,  and  energy  utilization  data  (AFCESA,  2007).  The  ACES  database  replaced  and 
captured  the  data  from  the  Planning,  Design,  and  Construction  (PDC)  system  previously 
used  to  manage  MILCON  projects  in  FY00  (Department  of  the  Air  Force,  2000).  AF 
regulation  requires  all  AF  MILCON  projects  be  tracked  and  managed  using  the  fields  in 
ACES.  Interviews  with  MILCON  managers  indicate  that  the  MILCON  ACES  data  is  up 
to  date  and  is  accurately  reflective  of  project  costs  as  it  represents  the  means  by  which 
funds  are  disbursed  and  contractors  are  paid  (Gogel,  2007a;  Langley,  2007b).  The 
ACES-PM  database  will  be  used  as  the  data  source  to  retrieve  project  infonnation  for  this 
study. 

Data  Collection 

Project  information  was  collected  from  ACES-PM  through  the  help  and  guidance 
of  the  construction  management  branch  of  Air  Combat  Command  and  Air  Force  Material 
Command.  Project  information  for  all  AF  MILCON  projects  from  FY  1990  to  2009  was 
retrieved  via  an  Oracle  Discoverer  report.  Managers  use  the  Oracle  Discoverer  report¬ 
writing  software  to  extract  data  from  ACES-PM  (PACAF,  2004).  The  data  retrieved 
from  the  Discoverer  report  was  placed  in  an  excel  spreadsheet  for  project  selection  and 
analysis  (Wells,  2007).  The  data  that  would  be  used  for  this  study  required  two 
additional  steps.  First,  project  selection  criteria  were  established  to  select  projects  within 
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the  scope  of  the  research.  Second,  the  ACES-PM  fields  that  contain  project,  cost,  and 
schedule  data  were  identified. 

Project  Selection  Criteria 

Projects  were  selected  from  ACES-PM  using  the  following  criteria: 

1 .  Only  continental  United  States  (CONUS)  projects  were  selected  to  avoid  effects 
from  overseas  costs. 

2.  Project  fiscal  year  (FY)  ranging  from  FY  1996  to  FY  2007  will  be  analyzed.  This 
year  range  will  enable  the  analysis  of  the  design-build  method  after  the  Clinger- 
Cohen  act  of  1996  opened  the  two-phase  method  for  use.  Additionally,  this  year 
group  will  build  on  previous  studies  (Webster,  1997)  that  ended  at  FY  1996. 

3.  Projects  meeting  the  minimum  project  value  at  the  MILCON  spending  level  will 
be  included.  This  level  was  $500,000  for  FY90  to  FY02  and  $750,000  for  FY03 
to  FY07. 

4.  All  projects  must  show  a  100%  construction  completion  level  in  ACES-PM. 

5.  This  research  will  analyze  MILCON  projects  in  the  321  funding  category. 
Emergency  341  MILCON  projects  are  not  included  due  to  the  unique  timelines 
and  methods  of  procurement. 

6.  Due  to  differences  in  funding  and  contracting  policies,  no  military  family  housing 
projects  will  be  included 

ACES-PM  Field  Selection 

The  ACES-PM  database  is  composed  of  many  fields  that  contain  a  multitude  of 
project  details.  The  fields  that  contain  project,  cost,  and  schedule  data  were  identified  in 
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order  to  retrieve  the  necessary  information  to  perform  a  comparative  analysis.  The 
AFMC  civil  engineering  construction  contract  (AFMC/A7CCC)  staff  aided  in  the 
selection  of  fields  that  were  incorporated  in  the  Discoverer  report.  The  complete  listing 
of  all  fields  incorporated  in  the  report  is  located  in  Appendix  A.  The  sample  size  will 
further  be  refined  by  removing  projects  with  a  “null”  or  “other”  listed  design  method, 
duplicate  projects,  projects  at  overseas  locations  that  slipped  through  the  criteria  of  the 
report,  and  projects  with  incomplete  data  fields. 

Data  Analysis 

Now  that  the  data  source  and  project  selection  criteria  have  been  established,  the 
method  in  which  the  data  will  be  analyzed  must  be  defined.  First,  the  performance 
metrics  to  be  used  by  this  study  will  be  defined.  Next,  the  method  in  which  the  data  is 
separated  into  facility  type  and  project  size  will  be  identified.  The  way  in  which  the 
delivery  methods  are  compared  over  time  will  be  described.  Finally,  the  statistical 
method  that  will  be  used  to  determine  significance  over  the  performance  metrics,  facility 
types,  and  timeline  will  be  defined. 

Performance  Metrics 

The  literature  identified  performance  metrics  used  by  AF  project  managers  and 
previous  studies  to  compare  project  delivery  methods.  Eight  performance  metrics  will  be 
used  by  this  study  and  include:  unit  cost,  cost  growth,  schedule  growth,  construction 
speed,  modifications  per  million  dollars,  CWE/PA  ratio,  construction  timeline,  and  total 
project  time.  Equations  were  developed  for  the  project  data  retrieved  from  ACES-PM  to 
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calculate  the  performance  metrics.  Each  equation  was  reviewed  and  commented  on  by 
AF  project  managers. 


Unit  Cost 

The  unit  cost  indicates  the  dollar  amount  per  unit  of  area  of  the  project  cost.  Unit 
cost  typically  consists  of  the  funds  used  to  construct  the  physical  building  excluding 
supporting  facilities,  equipment,  supervision,  inspection,  and  overhead  costs  (SIOH) 
(UFC,  2007).  This  study  will  use  the  Total  Current  Working  Estimate  (CWE)  amount 
when  calculating  unit  cost.  The  CWE  includes  all  of  the  costs  associated  with  the  project 
including  modification,  contingency,  SIOH,  design,  and  management  reserve  costs.  The 
equation  for  unit  cost  for  this  study  is: 

Unit  Cost 

where 

Total  CWE  Amt  =  Total  Current  Working  Estimate  Amount  ($) 

Index  =  time  adjustment  *  location  adjustment 
Scope  =  quantity  of  units  constructed  (m2) 

The  CWE  indicates  the  final  project  cost  since  all  projects  in  the  analysis  were 
100%  construction  complete.  An  index  factor  was  calculated  in  order  to  compare 
projects  constructed  in  different  locations  and  years.  The  time  adjustment  factor  used  the 
RSMeans  (2006)  National  30-city  average  historical  escalation  factor.  The  time  factor 
was  calculated  by  dividing  the  national  average  for  2006  by  the  national  average  for  the 


$  )  (Total  CWE  Amt  *  Index) 
nr )  Scope 
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year  of  the  project.  The  MILCON  Area  Cost  Factor  (ACF)  was  used  to  account  for 
project  location  as  published  in  the  DoD  Facilities  Pricing  Guide  (UFC,  2007). 

Cost  Growth 

Cost  growth  indicates  the  percent  difference  between  the  original  contract  cost 
and  the  actual  contract  cost  (Pocock,  1996).  The  AF  MILCON  Dirtkicker  goal  is  to  have 
the  ratio  of  current  contract  cost  to  original  contract  cost  be  less  than  5%  (Fox,  2006). 
Increases  to  the  original  contract  cost  are  documented  in  the  contract  modification 
amount  field  in  ACES-PM.  The  equation  for  cost  growth  is: 


Cost  Growth  (%)  = 


Contract  Mod  Amt 
Contract  Orig  Amt 


where 


Contract  Mod  Amt  =  contract  modification  amount  ($) 

Contract  Orig  Amt  =  contract  original  amount  ($) 

Contract  modifications  originate  from  many  different  sources.  Modifications  can 
indicate  problems  with  the  design  or  construction  errors,  environmental  or  unforeseen  site 
conditions,  or  might  not  be  problems  at  all  and  indicate  additional  benefits  added  to  the 
project  (Langley,  2007b).  An  in-depth  analysis  of  modifications  and  change  orders  is 
beyond  the  scope  of  this  thesis.  A  limitation  of  this  study  is  that  the  cause  of  a 
modification  is  not  indicated  in  ACES-PM;  therefore,  this  study  will  assume  that  a 
modification  is  generated  from  a  problem  or  negative  cause. 
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Schedule  Growth 


The  farther  a  project  continues  past  its  planned  completion  date,  the  greater  the 
impact  to  the  cost  of  the  project  and  delays  occupancy  or  use  of  the  facility.  Keeping 
schedule  growth  to  a  minimum  is  a  goal  of  every  project  manager.  The  equation  to 
calculate  schedule  growth  from  the  ACES-PM  data  is: 


Schedule  Growth  (% 


"NTP  to  Act  BOD 
v  NTP  to  Est  BOD 


-  1 


100 


(3) 


where 

NTP  to  Act  BOD  =  notice  to  proceed  to  actual  beneficial  occupancy  date 

NTP  to  Est  BOD  =  notice  to  proceed  to  estimated  beneficial  occupancy  date 
Negative  values  indicate  an  early  project  finish.  The  AF  MILCON  Dirtkicker  goal  is  to 
keep  schedule  growth  less  than  10%  (Fox,  2006).  The  ACES-PM  data  for  NTP  to  BOD 
is  the  construction  phase  of  traditional  projects  but  is  design  and  construction  for  design- 
build  projects.  The  traditional  design  schedule  growth  is  not  documented  in  ACES-PM. 
Therefore,  interpretation  of  the  results  of  the  schedule  growth  metric  must  be  conducted 
with  the  understanding  of  this  limitation. 

Construction  Speed 

Construction  speed  is  an  indicator  of  how  quickly  the  project  team  was  able  to 
deliver  the  facility  (Konchar  &  Sanvido,  1998).  The  construction  speed  value  indicates 
the  amount  of  square  footage  per  month  that  was  constructed.  The  Konchar  &  Sanvido 
(1998)  equation  was  adapted  for  ACES-PM  MILCON  data  and  is  represented  as: 
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Construction  Speed  (nr/month)  =  T - 

F  f  NTP  to  Act  BOD 

,  30 


(4) 


where 

Scope  =  number  of  units  constructed  (m  ) 

NTP  to  Act  BOD  =  notice  to  proceed  to  actual  beneficial  occupancy  date 
Modifications  per  Million  Dollars 

The  number  of  modifications  per  million  dollars  was  used  by  Pocock  (1996)  and 
Webster  (1997)  as  an  indirect  measure  indicating  how  many  problems  the  project 
experienced.  This  indicator  is  used  to  quantitatively  compare  the  typically  subjective 
quality  performance  of  the  design-build  and  traditional  delivery  methods.  The  limitations 
of  using  ACES-PM  data  previously  discussed  in  the  cost  growth  section  also  apply  to  this 
success  metric.  The  equation  to  determine  the  modifications  per  million  dollars  was 


Modifications 


Mod  Count  Qty 

Contract  Orig  Amt  +  Eng  Dsg  Amt 

$1,000,000 


(5) 


where 


Mod  Count  Qty  =  number  of  modifications 
Contract  Orig  Amt  =  contract  original  amount  ($) 

Eng  Dsg  Amt  =  engineering  design  amount  ($) 

Project  cost  is  calculated  only  using  the  cost  for  design  and  construction  contracts 
for  the  MILCON  project.  Additional  costs  are  with  held  in  order  to  detennine 
modifications  resulting  from  the  delivery  method  costs.  Dividing  the  number  of 
modifications  by  the  contract  value  normalizes  the  effects  of  project  size  (Pocock,  1996). 
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In  addition  to  metrics  identified  by  previous  studies,  several  metrics  were  chosen 
for  comparison  of  the  delivery  methods  from  the  Dirtkicker  criteria.  Along  with  the 
previously  mentioned  cost  and  schedule  growth,  the  CWE/PA  ratio  and  construction 
timeline  metrics  will  be  investigated. 

CWE/PA  Ratio 


The  CWE/PA  ratio  is  representative  of  the  effectiveness  of  the  delivery  method  in 
meeting  the  programmed  amount  of  MILCON  projects.  The  equation  for  the  ratio  is: 


CWE/PA  = 


Total  CWE  Amt 

yA 


(6) 


where  PA  is  the  programmed  amount.  The  award  CWE  cannot  exceed  25%  or  $2.0 
million,  whichever  is  greater,  of  the  PA  without  being  redesigned,  re-bid,  or 
reprogrammed  (Department  of  the  Air  Force,  2000).  The  AF  goal  is  to  have  a  CWE/PA 
ratio  less  than  or  equal  to  0.97  (Fox,  2006). 

Construction  Timeline 

The  Dirtkicker  criteria  defines  construction  timeline  as  “a  measure  of  how  often 
projects  meet  acceptable  performance  time  targets  based  on  the  Programmed  Amounts” 
(Fox,  2006).  The  equation  used  by  this  study  to  calculate  this  metric  is: 

Construction  Timeline  (days)  =  (NTP  to  Act  BOD)  -  Target  Days  (7) 

where  target  days  are: 

365  Days  for  PA  <  $5M 

540  Days  for  PA  >  $5M  and  PA  <  $20M 

730  Days  for  PA  >  $20M 
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Negative  values  indicate  the  project  finishing  earlier  than  the  Dirtkicker  goal. 
MAJCOMs  are  graded  on  the  percentage  of  projects  that  meet  the  target  day  goal  (Fox, 
2006).  Construction  timeline  was  selected  since  the  Dirtkicker  requirement  is  used  as  the 
standard  when  writing  RFP  and  IFB  packages  (Langley,  2007b).  Again,  the  ACES-PM 
data  places  a  limitation  on  the  direct  delivery  method  comparison  of  construction 
timeline.  NTP  to  BOD  is  the  construction  phase  of  traditional  projects  but  is  design  and 
construction  for  design-build  projects.  The  traditional  design  timeline  is  documented  in 
ACES-PM  with  a  design  start  date  but  without  a  design  completion  date.  Therefore,  the 
study  was  unable  to  add  design  time  to  the  duration  of  construction  for  traditional 
projects.  Therefore,  interpretation  of  the  results  of  the  construction  timeline  metric  must 
be  conducted  with  the  understanding  that  the  design-build  construction  timeline  includes 
design  and  construction  whereas  traditional  construction  timeline  results  only  include 
construction. 

Total  Project  Time 

The  final  performance  metric  results  from  the  interest  in  total  project  time  by  AF 
project  managers  (Astin,  2007;  Langley,  2007b;  Pocock,  2007a).  This  metric  has  been 
avoided  in  research  due  to  the  uncertainty  in  AF  MILCON  programming  and  ACES-PM 
documentation  (Astin,  2007;  Langley,  2007b).  Total  project  time  refers  to  the  moment 
design  action  begins  on  a  project  to  when  the  occupants  enter  the  building.  The  equation 
for  total  project  time  is: 

Total  Project  Time  (days)  =  Actual  BOD  -  Field  DI  Issued  (8) 
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where  Field  DI  Issued  is  the  date  when  the  program  managers  at  MAJCOM  direct  the 
agent  (ACOE,  NAVFAC,  AFCEE)  to  begin  design  of  the  project.  The  Actual  BOD  date 
is  when  the  user  takes  occupancy  of  the  facility.  The  difference  between  these  dates  will 
provide  project  managers  the  answer  to  which  delivery  method  is  faster  from  start  to 
finish.  Figure  9  was  developed  in  order  to  display  the  schedule  performance  metrics.  It 
shows  the  MILCON  project  milestones  and  where  those  milestones  correspond  with  the 
activities  of  each  project  delivery  method.  Figure  9  also  displays  the  limitations  placed 
on  the  schedule  growth  and  construction  timeline  comparisons  by  the  MILCON  ACES- 
PM  documentation  for  this  study. 


Traditional 


Design-Build 


1  st  Contract 

a-e  Selection  Award  Design 

1  2nd  Contract 

Advertise 

Award 

Construction 

dj 

Single 

Contract 

Develop  RFP 

Issue  RFP 

Award 

Design  &  Construction 

Figure  9.  Schedule  Performance  Metric  Diagram 
Historical  Delivery  Method  Performance 

The  data  selected  provides  the  ability  to  observe  the  historical  trends  of  the 
design-bid-build  and  design-build  delivery  methods.  Webster  (1997)  attempted  to 
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determine  if  the  design-build  method  had  improved  from  the  study  by  Pocock  (1996). 
However,  a  sufficient  track  record  of  projects  was  not  available  to  either  study  due  to  the 
new  use  and  approval  of  the  delivery  method  and  the  lack  of  a  centralized  database  like 
ACES-PM  to  track  MILCON  projects.  This  study  will  attempt  to  capture  all  AF 
MILCON  projects  since  design-build  was  approved  for  widespread  use  in  1996 
(Loulakis,  2003). 

The  selected  data  will  be  grouped  into  five  two-year  categories  and  compared 
within  the  study  to  determine  if  the  design-build  method  has  improved  through  time.  The 
first  group  will  include  data  from  FY  1996  to  FY  1997.  These  projects  occurred  after  the 
Clinger-Cohen  act  of  1996  authorized  the  Two-step  design-build  method  for  all 
MILCON  projects  (Loulakis,  2003).  The  remaining  projects  will  be  grouped  into  year 
groups  FY  1998  to  FY  1999,  FY  2000  to  FY  2001,  FY  2002  to  FY  2003,  and  FY  2004  to 
FY  2005.  The  time  study  will  end  with  FY  2005  projects  because  at  the  time  of  this 
study,  few  FY  2006  and  even  fewer  FY  2007  MILCON  projects  would  be  100% 
construction  complete. 

Facility  Type 

The  Blue  Book  (Department  of  the  Air  Force,  2000)  suggested  certain  facility 
types  for  use  with  the  design-build  method  due  to  available  private  sector  standards, 
commercial  standards,  and  AF  design  guides  (Table  8).  The  facility  type  will  be 
determined  from  the  retrieved  ACES-PM  data  from  the  Category  Code  (CATCODE). 

The  CATCODE  is  a  six-digit  number  that  represents  the  function  of  the  facility  or  area  of 
the  project  (Department  of  the  Air  Force,  1996).  The  first  two  numbers  of  the 
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CATCODE  identify  the  main  facility  type  for  the  project.  The  CATCODE  for  each 
facility  type  is  displayed  in  Table  12. 


Table  12.  Facility  Ca 


egory  Code  Prefix  (Department  of  the  Air  Force,  1996) 


CATCODE 

Description 

11 

Airfield  Pavements 

12 

Petroleum  Dispensing  and  Operating  Facilities 

13 

Communications,  Navigation  Aids  &  Airfield  Lighting 

14 

Land  Operations  Facilities 

17 

Training  Facilities 

21 

Maintenance  Facilities 

41 

Liquid  Fuel  Storage 

42 

Explosives  Facilities 

44  &  45 

Storage  Facilities  Covered,  Open,  Special  Purpose 

61 

Admin  Facilities 

72 

Dorms,  Officer  Quarters,  Dining  Halls 

73 

Personnel  Support 

81 

Electricity 

82  &  83  &  84 

Heat,  Sewage  &  Waste,  Water 

85 

Roadway  Facilities 

86 

Railroad  Trackage 

87 

Ground  Improvement  Structures 

88 

Fire  and  Other  Alarm  Systems 

89 

Miscellaneous  Utilities 

It  was  arbitrarily  detennined  that  at  least  10  projects,  5  design-build  and  5  design- 
bid-build,  must  be  retrieved  from  ACES-PM  for  each  facility  type  in  order  to  compare 
the  delivery  method  according  to  the  performance  metrics.  The  best  delivery  method  for 
a  facility  will  be  determined  by  the  majority  of  significant  findings  by  a  particular 
delivery  method  for  that  facility  type. 
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Statistical  Analysis 

This  study  will  use  statistics  to  test  if  the  design-build  sample  performance 
metrics  are  significantly  different  than  the  design-bid-build  performance  metrics.  This 
study  will  use  the  /-test  for  samples  with  unequal  variances  method  to  compare  two 
population  means  (Pocock,  1996;  Webster,  1997;  McClave,  2005).  Two  conditions  must 
be  met  in  order  for  the  two  sample  t-test  to  be  valid.  First,  the  samples  must  be 
independent  and  random  samples  from  each  population.  Secondly,  the  sample 
populations  must  have  distributions  that  are  approximately  normal  (McClave,  2005). 

Once  these  conditions  are  met,  the  t-test  will  determine  what  the  p-value  for  that 
comparison.  The  Microsoft  Excel  Data  Analysis  package  will  be  used  to  compute  the  p- 
values  for  analysis. 

Study  Hypothesis 

For  the  perfonnance  metrics  defined,  the  facility  types  identified,  and  the  design- 
build  method  improvement  over  time: 

•  The  null  hypothesis  for  this  study  is:  There  will  be  no  significant  difference 
between  the  between  the  average  perfonnance  of  the  design-bid-build  and  the 
design-build  delivery  method. 

•  The  alternative  hypothesis  is:  The  average  performance  of  the  design-build 
method  will  be  significantly  better  than  the  design-bid-build  approach. 

Study  Significance  Level 

The  level  of  significance,  a ,  represents  a  Type  I  error  where  the  null  hypothesis  is 
rejected  when  in  fact  the  null  hypothesis  was  true.  The  hypothesis  test  statistic  (t  value) 
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produces  an  observed  significance  level  for  the  sample,  or  p-value,  that  is  compared  to 
the  detennined  a  level.  A  p-value  less  than  a  for  a  one-tailed  test  reduces  the  likelihood 
that  the  null  hypothesis  was  rejected  in  error  (Pocock,  1996;  McClave,  2005).  For  this 
study,  a  one-tailed  test  statistic  that  produces  a  p-value  less  than  0.05  will  be  considered 
statistically  significant  and  highly  significant  if  less  than  0.01  (Webster,  1997). 

Summary 

Chapter  3,  Methodology,  described  the  data  source,  data  collection,  and  data 
analysis  methods  that  will  be  used  to  compare  the  delivery  methods  used  for  MILCON. 
The  design-build  delivery  method  will  be  compared  with  the  design-bid-build  method 
over  eight  performance  metrics,  over  time,  and  according  to  facility  type.  Chapter  4, 
Results,  will  now  present,  analyze,  and  discuss  the  data  in  order  to  test  the  study 
hypotheses  and  answer  the  research  questions. 
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IV.  Analysis  and  Results 


Chapter  Overview 

This  chapter  provides  the  raw  results  of  the  study  based  on  the  methodology 
developed  in  Chapter  3.  The  results  are  presented  graphically  and  in  tables  with 
discussions  limited  to  the  statistical  analysis.  Additional  interpretation,  explanation,  and 
speculation  of  the  results  will  be  addressed  in  Chapter  5,  Conclusions. 

The  first  section  describes  the  data  retrieved  from  the  Automated  Civil  Engineer 
System  -  Project  Management  Module  (ACES-PM)  used  in  this  analysis.  The  second 
section  compares  the  performance  of  design-build  and  design-bid-build  MILCON 
projects,  using  all  the  data  collected,  across  eight  performance  metrics:  unit  cost,  cost 
growth,  schedule  growth,  construction  speed,  modifications  per  million  dollars,  CWE/PA 
ratio,  construction  timeline,  and  total  project  time.  The  performance  over  time  for  each 
metric  is  then  analyzed.  Finally,  the  data  is  grouped  according  to  facility  type  and 
analyzed  across  the  eight  performance  metrics.  A  summary  of  the  total  analysis  results  is 
provided  in  Table  14  on  page  95. 

Results 

The  initial  data  retrieved  from  ACES-PM  was  used  to  observe  the  percentage  of 
the  AF  MILCON  program  that  utilized  the  design-build  delivery  method.  All  AF 
MILCON  projects  were  gathered  from  FY  1990  to  FY  2009,  including  overseas  projects. 
Figure  10  displays  the  results  of  this  initial  observation.  The  use  of  the  design-build 
method  began  to  increase  after  the  Clinger-Cohen  Act  of  1996  allowed  the  unrestricted 
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AF  MILCON  1990-2009 


Figure  10.  MILCON  Delivery  Method  Use 
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use  of  design-build  for  MILCON.  The  most  notable  increase  observed  was  the  18%  use 
in  FY1999  rising  to  48%  for  FY2000.  The  actual  use  of  design-build  for  MILCON 
generally  matches  the  predictions  made  by  the  DBIA  for  the  industry  shown  in  Figure  3. 
Design-build  is  expected  to  overtake  the  traditional  method  for  FY2008  projects  at  53%, 
when  the  DBIA  predicted  this  to  occur  in  2010.  The  future  AF  MILCON  design-build 
use  is  expected  to  increase.  MILCON  project  management  was  transferred  from  the 
MAJCOMs  to  the  Air  Force  Center  for  Engineering  and  the  Environment  (AFCEE)  in 
October  2007  (AFCEE,  2007b).  AFCEE  will  now  use  the  design-build  delivery  method 
as  the  default  method  for  MILCON  execution  for  new  construction.  The  traditional 
method  will  be  used  primarily  for  MILCON  renovation  projects  (Morrison,  2007a). 
Therefore,  the  use  of  design-build  by  the  Air  Force  will  continue  to  grow  and  follow  the 
predictions  made  by  the  DBIA. 

Data  Description 

After  the  observation  of  MILCON  delivery  method  use,  the  project  data  was 
filtered  in  order  to  meet  the  project  selection  criteria  outlined  in  the  methodology.  The 
data  retrieved  needed  additional  refinement  in  order  to  remove  overseas  projects  that 
were  incorrectly  coded  and  projects  with  insufficient  field  population.  835  projects  met 
the  required  analysis  criteria  consisting  of  557  (67%)  traditional  design-bid-build  and  278 
(33%)  design-build  projects. 

In  order  to  proceed  with  the  statistical  analysis,  the  conditions  required  to  ensure  a 
valid  t-test  had  to  be  met.  The  first  requirement  was  that  the  data  must  be  selected 
independently  and  randomly.  The  data  used  in  this  study  met  these  criteria  because  every 
possible  data  point  was  retrieved  without  a  preference  to  any  particular  project  type  and 
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without  choosing  particular  projects  for  analysis.  Additionally,  every  project  in  the 
sample  was  independent  of  each  other  with  regards  to  the  performance  metrics  chosen. 
Secondly,  the  sampled  populations  needed  to  have  a  distribution  that  was  approximately 
nonnal.  The  design-build  and  traditional  results  for  each  perfonnance  metric  was  plotted 
on  a  histogram  located  in  Appendix  K.  The  results  of  these  histograms  show  that  the 
design-build  and  traditional  distributions  for  the  performance  metric  were  approximately 
nonnal  and  satisfy  the  second  requirement  for  perfonning  a  two-sample  t-test  with 
unequal  variances. 

Performance  Metric  Analysis 

The  performance  metric  analysis  compared  the  entire  sample  of  835  projects 
across  eight  performance  metrics.  Table  13  displays  the  overall  results  of  the  one  tailed  t- 
test  for  the  comparison  between  design-build  and  design-bid-build  projects.  The  mean, 
variance,  number  of  observations,  and  p-value  was  calculated  for  design-build  and 
traditional  delivery  methods  for  each  metric.  As  outlined  in  the  methodology,  a  p-value 
of  less  than  0.05  indicates  statistical  significance  and  less  than  0.01  is  highly  significant. 
The  p-value  indicates  the  level  at  which  the  null  hypothesis  is  rejected  in  error.  Each 
performance  metric  will  now  be  discussed  including  the  historic  perfonnance 
characteristics  of  the  design-build  delivery  method.  The  explanatory  results,  provided  by 
Air  Force  project  managers,  of  the  historical  analysis  will  be  reserved  for  the  discussion 
located  in  Chapter  5:  Conclusions. 
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Table  13.  Overall  Analysis  Results 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

3041.0872 

2706.5277 

6403431.1 

4179979.2 

111 

356 

0.0637538 

Cost  Growth(%) 

4.5145524 

6.4186242 

38.104292 

233.67265 

111 

553 

0.0055705 

Schedule  Growth  (%) 

17.341071 

18.819136 

1135.8948 

1604.5974 

271 

507 

0.2929871 

Construction  Speed  (SM/Month) 

422.97993 

683.17434 

1518378 

10927894 

176 

351 

0.0962699 

Modifications  Per  $M  (#/$M) 

2.0999542 

4.7975433 

8.3208965 

49.418071 

276 

557 

7.98E-15 

CWE/PA 

0.9904743 

0.9907844 

0.0130928 

0.0521008 

278 

557 

0.4895706 

Construction  Timeline 
(#  days  over  target) 

PA  <  $5M 

195.05797 

155.44523 

55568.467 

43357.73 

69 

283 

0.1019087 

S5M  <  PA  <  $20M 

161.59794 

98.665807 

64792.387 

50821.261 

194 

243 

0.0036144 

PA  >  $20M 

202.75 

56.375 

72454.386 

20856.517 

12 

16 

0.0534474 

Total  Project  Time  (Years) 

3.2820922 

3.0530874 

1.4117642 

1.8354118 

275 

546 

0.0066169 

Unit  Cost 

The  traditional  average  unit  cost  was  better  than  design-build  at  $2,706.53/m2  and 
$3,041.09/nr,  respectively.  However,  the  p-value  of  0.0637  failed  to  reject  the  null 
hypothesis  and  the  difference  is  not  significant.  The  sample  size  used  to  compare  unit 
cost  consisted  of  177  design-build  and  356  traditional  projects.  Project  removal  resulted 
from  dormitory  projects  listing  scope  in  number  of  rooms  or  people,  other  projects  not 
measured  in  square  meters,  and  unpopulated  ACES-PM  CWE  and  scope  fields.  A  unit 
cost  analysis  was  conducted  later  in  the  facility  type  analysis  using  the  number  of  rooms 
as  the  scope. 

Figure  1 1  displays  the  unit  cost  performance  of  design-build  and  traditional  AF 
MILCON  projects  over  time.  The  only  significant  difference  in  delivery  methods 
occurred  in  the  FY00-01  year  group.  The  traditional  ($2,166.77/m2)  method 
outperformed  design-build  ($2, 932. 06/m2)  with  a  highly  significant  p-value  of  0.0028. 
The  unit  cost  performance  for  design-build  worsened  over  time.  The  FY02-03  unit  cost 
was  worse  than  the  previous  year  group  FY00-01  at  a  significant  level  of  p=0.041  and 
when  compared  to  the  starting  year  group  FY96-97  at  a  significant  level  of  p=0.01 1.  The 
complete  t-test  results  for  the  time  comparison  of  unit  cost  are  provided  in  Appendix  B. 
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Figure  11.  Unit  Cost  Performance  over  Time 


Firm  conclusions  cannot  be  drawn  from  the  unit  cost  analysis  until  facility  type 
has  been  accounted  for.  The  previous  analysis  included  project  from  every  facility  type. 
The  facilities  ranged  from  airfield  pavement  projects  to  fitness  centers  to  maintenance 
hangars  and  research  and  development  labs,  all  of  which  have  significantly  different  costs 
per  square  meter.  Additionally,  the  unit  cost  variation  over  time  may  result  from  many 
projects  of  a  certain  type  of  facility  being  constructed  for  a  particular  year.  For  example, 
the  FY00-01  group  had  12  traditional  pavement  projects  compared  to  three  that  used 
design-build.  Conclusive  results  for  unit  cost  will  arise  from  the  facility  type  analysis 
described  later  in  this  chapter. 

Cost  Growth 

The  cost  growth  analysis  showed  that  the  design-build  delivery  method 
outperformed  the  traditional  method  with  a  p-value  of  p=0.0056,  indicating  a  difference 
between  the  two  delivery  methods  at  a  highly  significant  level.  The  design-build  mean 
cost  growth  was  4.5 1%  while  the  traditional  average  was  higher  at  6.41%.  The  sample 
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size  used  for  the  cost  growth  analysis  contained  277  design-build  and  553  traditional 
projects.  Projects  were  removed  from  the  analysis  that  did  not  have  values  in  the  original 
contract  amount  field  of  ACES-PM. 

The  historical  analysis  of  cost  growth  yielded  several  notable  results.  Figure  12 
displays  the  cost  growth  performance  of  design-build  and  traditional  AF  MILCON 
projects  over  time.  Although  not  statistically  significant,  design-build  consistently 
outperformed  the  traditional  method  for  every  year  group.  The  design-build  delivery 
method  itself  improved  over  time.  The  FY04-05  (2.79%)  design-build  cost  growth 
improved  from  FY02-03  (4.84%)  at  a  significant  level  of  p=0.022  and  from  FY96-97 
(8.21%)  at  a  highly  significant  level  of  p=0.0024. 


Cost  Growth  (%) 


-  Design  Build 
■  Traditional 


Figure  12.  Cost  Growth  Performance  over  Time 


The  traditional  delivery  method  performance  also  improved  over  time.  The 
FY04-05  (3.25%)  traditional  cost  growth  improved  at  a  highly  significant  level  of 
p=6.6xl0';’  when  compared  to  the  FY96-97  (8.26%)  year  group.  Meanwhile,  the 
traditional  comparisons  between  each  year  group  did  not  result  in  any  statistically 
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significant  results.  The  t-test  results  for  the  time  comparison  of  cost  growth  are  provided 
in  Appendix  C.  The  cost  growth  study  showed  that  the  design-build  delivery  method 
outperforms  the  traditional  design-bid-build  method  and  has  improved  over  time  at  a 
statistically  high  significant  level. 

Schedule  Growth 

The  schedule  growth  analysis  detennined  that  the  mean  schedule  growth  was 
17.3%  for  design-build  projects  and  18.8%  for  traditional  projects.  While  the  design- 
build  method  had  lower  schedule  growth,  the  results  of  the  t-test  were  not  nearly 
significant  due  to  a  p-value  of  0.29.  The  sample  size  for  the  analysis  of  schedule  growth 
consisted  of  271  design-build  and  507  traditional  projects.  The  sample  size  was  reduced 
by  removing  projects  with  missing  data  in  the  notice  to  proceed  (NTP)  and  beneficial 
occupancy  date  (BOD)  fields  in  ACES-PM.  In  several  cases,  the  revised  BOD  field  was 
used  instead  of  the  estimated  BOD  field  when  the  duration  of  NTP  to  estimated  BOD  was 
a  zero  or  negative  number. 

The  historical  analysis  shown  in  Figure  13  shows  a  trend  of  increasing  schedule 
growth  for  both  design-build  and  traditional  delivery  method.  The  Dirtkicker  goal  for 
schedule  growth  is  to  have  projects  below  10%  growth.  The  design-build  schedule 
growth  increased  from  FY96-97  to  FY00-01  at  a  significant  level  with  p=0.023.  While 
the  overall  comparison  between  design-build  and  traditional  methods  was  not  significant, 
there  was  a  significant  difference  in  the  FY02-03  group.  The  FY02-03  schedule  growth 
for  design-build  (17.37%)  was  better  than  traditional  (33.69%)  at  a  highly  significant  p- 
value  of  0.002.  The  t-test  results  for  the  time  comparison  of  schedule  growth  are 
provided  in  Appendix  D. 
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A  limitation  for  the  schedule  growth  analysis  is  based  on  the  project 
documentation  in  ACES-PM.  The  only  fields  that  lend  themselves  to  schedule  analysis 
are  the  NTP  and  BOD  milestones.  NTP  to  BOD  only  accounts  for  the  construction 
portion  of  traditional  projects  while  NTP  to  BOD  for  design-build  includes  both  design 
and  construction  phases.  The  results  already  show  that  design-build  had  a  lower  schedule 
growth  than  the  traditional  method.  Although  not  investigated  by  this  study,  it  can  be 
expected  that  the  schedule  growth  for  traditional  projects  would  increase  once  the  design 
schedule  was  included. 

Construction  Speed 

The  traditional  construction  speed  was  faster  than  design-build  at  683.2  mVmonth 
and  422.9  m  /month,  respectively.  However,  the  p-value  of  0.096  failed  to  reject  the  null 
hypothesis  and  the  difference  is  not  significant.  The  sample  size  used  to  compare  unit 
cost  consisted  of  176  design-build  and  35 1  traditional  projects.  As  with  the  unit  cost 
metric,  all  dormitory  projects  were  removed  from  this  analysis  due  to  the  listing  scope  in 
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number  of  rooms  or  people  instead  of  square  meters.  Additional  projects  were  removed 
due  to  the  project  scope  not  being  measured  in  square  meters  and  unpopulated  ACES-PM 
NTP,  BOD,  and  scope  fields. 


Figure  14  displays  the  construction  speed  performance  of  design-build  and 
traditional  AF  MILCON  projects  over  time.  The  complete  t-test  results  for  the  time 
comparison  of  unit  cost  are  provided  in  Appendix  E.  The  construction  speed  over  time 
analysis  did  not  yield  any  statistically  significant  results.  The  comparison  within  design- 
build  over  time  was  not  significant,  neither  was  the  comparison  between  design-build  and 
traditional  for  each  two  year  group. 


Construction  Speed 


— « — Design  Build 
— ■ — Traditional 


Figure  14.  Construction  Speed  Performance  over  Time 


As  with  the  unit  cost  metric,  firm  conclusions  cannot  be  drawn  from  the 
construction  speed  analysis  until  facility  type  has  been  accounted  for.  The  impact  of 
pavement  projects  was  even  greater  for  construction  speed.  The  large  spike  in  the  FY00- 
01  traditional  group  construction  speed  was  a  direct  result  of  a  FY2000  94,500  m  airfield 
upgrade  project  for  Tyndall  AFB  with  a  construction  speed  of  8,03 1.16  m  /month  and  a 
FY2001  220,244  m2  auxiliary  airfield  project  for  Charleston  AFB  with  a  construction 
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speed  of  40,288.54  m  /month.  These  large  airfield  projects  were  the  cause  for  the  lack  of 
significance  with  the  traditional  method  construction  speed.  Pavement  projects  are 
included  in  the  facility  analysis  and  will  be  discussed  later. 

Modifications  per  Million  Dollars 

The  design-build  delivery  method  outperformed  the  traditional  method  with  2.7 
less  modifications  per  million  dollars  (Mods/$M)  at  a  highly  significant  level  of  p  = 

8x1  O'15.  The  design-build  mean  Mods/$M  was  only  2.09  while  the  traditional  average 
was  higher  at  4.79.  The  sample  size  used  for  the  cost  growth  analysis  contained  276 
design-build  and  557  traditional  projects.  Projects  were  removed  from  the  analysis  that 
did  not  have  values  in  the  modification  number,  original  contract  amount,  or  engineering 
design  amount  fields  of  ACES-PM. 

The  historical  analysis  of  Mods/$M  produced  highly  significant  findings.  Figure 
15  displays  the  Mods/$M  performance  of  design-build  and  traditional  methods  over  time. 
The  design-build  method  showed  significant  improvement  from  2.96  Mods/$M  in  FY96- 
97  to  2.1 1  Mods/$M  in  FY00-01  and  highly  significant  improvement  at  1.79  Mods/$M  in 
FY02-03  and  1.18  Mods/$M  in  FY04-05.  The  design-build  method  outperformed  the 
traditional  method  at  a  significant  level  for  FY00-01  and  at  a  highly  significant  level  for 
all  other  year  groups.  While  the  design-build  method  had  better  performance,  the 
traditional  delivery  method  displayed  more  improvement  over  time.  The  FY04-05  (2.23) 
traditional  Mods/$M  improved  at  a  highly  significant  level  of  p=1.4xl0"8  when  compared 
to  the  starting  FY96-97  (5.7)  year  group  and  a  significant  level  p=0.0109  when  compared 
to  FY02-03  (3.7).  The  t-test  results  for  the  time  comparison  of  modifications  per  million 
dollars  are  provided  in  Appendix  F.  The  modification  per  million  dollar  analysis  showed 
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that  the  design-build  delivery  method  outperforms  the  traditional  design-bid-build 
method  and  has  improved  over  time  at  a  highly  statistically  significant  level. 


Modifications  Per  Million  Dollars 
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Figure  15.  Modifications  per  $M  Performance  over  Time 


CWE/PA  Ratio 

The  current  working  estimate  (CWE)  to  programmed  amount  (PA)  ratio  analysis 
showed  that  there  is  practically  no  difference  in  the  performance  of  either  delivery 
method  for  this  metric.  The  p-value  of  p=0.489  confirms  that  there  is  no  difference 
between  the  CWE/PA  ratio  for  design-build  (0.9904)  and  traditional  (0.9907).  The 
sample  size  for  CWE/PA  contained  278  design-build  and  557  traditional  MILCON 
projects.  Projects  were  removed  for  this  analysis  that  did  not  contain  values  in  the  CWE 
or  PA  fields  of  ACES-PM.  This  metric  contained  the  largest  sample  size  after  the 
filtering  of  projects  according  to  the  methodology.  All  other  metric  analysis  used  this 
sample  as  a  baseline  of  projects  in  order  to  have  as  many  projects  with  complete  project 
data  as  possible. 


85 


The  analysis  of  CWE/PA  over  time  did  not  produce  any  consistent  trend  in  the 


performance  of  design-build  or  traditional  delivery  methods.  Figure  16  shows  the 
performance  of  each  delivery  method  alternating  for  each  year  group.  The  only 
comparison  that  yielded  a  significant  result  was  the  design-build  FY02-03  comparison 
with  the  initial  FY96-97  year  group.  No  comparison  between  the  two  delivery  methods 
was  significant.  The  t-test  results  for  the  time  comparison  of  CWE/PA  are  provided  in 
Appendix  G.  Therefore,  the  design-build  method  has  not  improved  over  time  when 
compared  to  itself  and  the  traditional  method  for  the  CWE/PA  metric. 


Current  Working  Estimate  to  Programmed  Amount  Ratio 


-♦—Design  Build 
-■—Traditional 


Figure  16.  CWE/PA  Performance  over  Time 


Construction  Timeline 

The  construction  timeline  is  a  Dirtkicker  metric  solely  used  to  measure  AF 
MILCON  project  success.  This  Dirtkicker  metric  measures  the  NTP  to  BOD  for  a 
project.  The  issues  associated  with  using  the  NTP  to  BOD  dates  in  ACES-PM  previously 
discussed  for  schedule  growth  also  affect  the  construction  timeline  analysis.  The  NTP  to 
BOD  for  design-build  includes  both  the  design  and  construction  of  the  facility  whereas 
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the  traditional  method  only  included  the  construction  duration.  It  is  difficult  to  directly 
compare  the  two  delivery  methods  for  this  metric  using  ACES-PM  because  the  design 
duration  is  not  captured  for  traditional  projects.  Therefore,  this  analysis  will  proceed, 
using  NTP  to  BOD  and  comparing  that  duration  to  the  target  days,  with  an  understanding 
that  the  construction  timeline  includes  design  time  for  design-build  projects.  The  results 
for  each  programmed  amount  (PA)  grouping  will  now  be  presented. 

PA  <  $5M 

The  target  day  duration  for  MILCON  projects  below  five  million  dollars  is  365 
days  from  NTP  to  BOD.  The  traditional  delivery  method  had  fewer  days  over  the  target 
than  design-build  with  a  mean  of  155.4  days  and  195. 1  days  respectively.  However,  the 
p-value  of  0. 101  failed  to  reject  the  null  hypothesis  and  the  difference  is  not  significant. 
The  sample  size  used  to  compare  construction  timeline  consisted  of  69  design-build  and 
283  traditional  projects.  Projects  were  removed  due  to  unpopulated  ACES-PM  NTP  and 
BOD  fields. 

The  historical  analysis  for  construction  timeline  of  projects  less  than  $5M  yielded 
significant  design-build  delivery  results.  Figure  17  displays  the  trend  of  design-build 
projects  improving  over  time  and  overtaking  the  traditional  method.  The  FY00-01  (166.7 
days)  and  FY02-03  (172.3  days)  groups  improved  at  a  significant  level  of  p=0.023  and 
p=0.035,  respectively,  and  the  FY04-05  (56.07  days)  group  improved  at  a  highly 
significant  level  of  p=0.0028  when  compared  to  the  369.3  days  for  the  initial  FY96-97 
year  group.  While  not  statistically  significant,  the  FY02-03  and  FY04-05  design-build 
group  performed  better  than  the  traditional  projects  in  these  groups.  This  is  an  interesting 
finding  since  the  project  durations  included  design  time. 
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Construction  Timeline  PA<$5M 


—♦—Design-Build 
■  Traditional 


Figure  17.  Construction  Timeline  PA<$5M  over  Time 


The  historical  analysis  showed  the  traditional  method  improved  over  time.  The 
FY04-05  (63.6)  traditional  days  over  the  construction  timeline  target  of  365  days 
improved  at  a  highly  significant  level  of  p=0.003  when  compared  to  the  starting  FY96-97 
(172.5)  year  group  and  at  p=0.004  when  compared  to  FY02-03  (182.4).  The  t-test  results 
for  the  time  comparison  of  construction  timeline  of  projects  less  than  $5M  are  provided 
in  Appendix  H. 

While  the  entire  sample  for  this  PA  category  showed  the  traditional  method  to 
have  better  results,  the  historical  analysis  shows  the  design-build  method  has  improved 
and  out-perfonned  the  traditional  method.  The  design-build  delivery  method  is  able  to 
design  and  construct  the  project  with  fewer  days  over  the  365  day  target,  while  the 
traditional  method  only  constructs  the  project  in  this  time.  This  finding  shows  that 
design-build  now  produces  facilities  faster  than  the  traditional  method  for  programmed 
amounts  less  than  $5M. 
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$5M  <  PA  <  $20M 


The  construction  timeline  analysis  for  this  PA  range  resulted  in  several  significant 
results.  The  target  day  duration  for  MILCON  projects  between  five  and  twenty  million 
dollars  is  540  days  from  NTP  to  BOD.  The  traditional  delivery  method  had  fewer  days 
over  the  target  with  a  mean  of  98.6  days  while  the  design-build  average  was  161.6  days 
over.  This  comparison  showed  a  highly  significant  level  with  a  p-value  of  p=0.003.  The 
sample  size  used  to  compare  construction  timeline  consisted  of  194  design-build  and  243 
traditional  projects.  Projects  were  removed  due  to  unpopulated  ACES-PM  NTP  and 
BOD  fields. 

The  historical  analysis  identified  the  FY 00-0 1  year  group  as  having  the  most 
impact  on  the  overall  results  for  the  construction  timeline  analysis.  The  FY00-01 
comparison  was  highly  significant  (p=2.2xl0'5)  in  favor  of  the  traditional  method  as 
displayed  in  Figure  18.  However,  the  FY00-01  group  was  the  only  significant  difference 
between  the  design-build  and  traditional  methods.  The  design-build  construction 
timeline  performance  has  improved  over  time.  The  47.9  day  mean  for  FY04-05  design- 
build  projects  is  significantly  better  than  the  137.1  day  mean  for  FY02-03  (p=0.023)  and 
highly  significant  when  compared  to  the  initial  185.1  day  mean  for  FY96-97  (p=0.003). 
The  gap  between  traditional  and  design-build  decreased  from  a  difference  of  19.1  days 
for  FY02-03  to  10.3  days  for  FY04-05. 
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Construction  Timeline  $5M<PA<$20M 


—♦—Design-Build 
■  Traditional 


Figure  18.  Construction  Timeline  $5M<PA<$20M  over  Time 

The  historical  analysis  showed  the  traditional  method  improved  over  time. 
However,  the  only  significant  improvement  of  p=0.012  was  the  comparison  between  the 
FY04-05  (37.6)  and  FY02-03  (1 18.03)  year  groups.  As  previously  discussed  in  the 
PA<$5M  section,  the  design-build  method  designs  and  constructs  projects  with  similar 
days  over  the  540  target  as  the  traditional  method,  which  only  completes  the  construction 
portion.  The  t-test  results  for  the  time  comparison  of  this  construction  timeline  are 
provided  in  Appendix  I. 


PA  >  $20M 


The  target  day  duration  for  MILCON  projects  above  twenty  million  dollars  is  730 
days  from  NTP  to  BOD.  The  traditional  delivery  method  had  fewer  days  over  the  target 
than  design-build  with  a  mean  of  56.4  days  and  202.8  days,  respectively.  However,  the 
p-value  of  0.053  failed  to  reject  the  null  hypothesis  and  the  difference  is  not  significant. 
The  sample  size  used  to  compare  this  PA  level  for  construction  timeline  was  relatively 
small,  consisting  of  12  design-build  and  16  traditional  projects.  Projects  were  removed 
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due  to  unpopulated  ACES-PM  NTP  and  BOD  fields.  The  small  sample  size  for  projects 
greater  than  $20M  did  not  allow  for  a  historical  analysis  to  be  conducted. 

Total  Project  Time 

This  metric  will  attempt  to  measure  which  delivery  method  is  the  fastest  in  terms 
of  total  project  time.  This  total  project  time  was  calculated  using  the  date  the  field  design 
instruction  (DI)  was  issued  from  the  MAJCOM  to  the  design  agents  to  the  beneficial 
occupancy  date  (BOD).  An  example  for  this  metric  is  the  time  it  takes  from  when  the 
customer  (wing  commander,  user,  base  engineer)  identifies  the  need  for  a  facility  to  when 
that  customer  can  take  occupancy  of  it. 

The  mean  traditional  total  project  time  was  3.05  years  while  the  mean  design- 
build  total  project  time  was  3.28  years.  This  3.3  month  difference  was  highly  significant 
because  the  p-value  of  p=0.0066  rejected  the  null  hypothesis  that  there  was  no  difference 
between  the  two  delivery  methods.  The  sample  size  used  to  compare  total  project  time 
consisted  of  275  design-build  and  546  traditional  projects.  Project  removal  resulted  from 
unpopulated  ACES-PM  field  DI  issue  date  and  actual  BOD  fields. 

The  finding  that  the  traditional  method  has  a  highly  significant  shorter  total 
project  time  was  not  a  surprise  to  AF  project  managers.  The  cause  of  this  finding  is  not  a 
result  of  poor  design-build  time  perfonnance,  but  of  the  bureaucratic  MILCON  process. 
The  key  milestone  in  the  MILCON  process  is  the  signing  of  the  defense  bill  by  the 
President  that  funds  the  MILCON  projects  for  that  fiscal  year.  At  this  milestone,  a 
traditional  project  has  a  95-100%  design  complete  project  that  is  ready  for  construction. 
The  design-build  project  has  only  an  RFP  and  will  need  to  be  designed  and  constructed 
after  the  funding  arrives.  The  project  time  advantage  typically  seen  by  the  private  sector 
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design-build  with  a  single  contract  for  design  and  construction  is  hindered  by  the 
MILCON  process  funding  the  project  through  construction  dollars. 

While  the  entire  sample  for  total  project  time  showed  the  traditional  method  to 
have  better  results,  the  historical  analysis  shows  the  design-build  method  has  improved. 
Figure  19  displays  the  historical  analysis  for  total  project  time.  The  traditional  method 
was  significantly  better  than  design-build  for  only  two  time  groups,  FY96-97  (p=0.018) 
and  FY00-01  (p=0.0005).  The  trend  in  the  design-build  performance  has  improved  at  a 
highly  significant  level  (p=0.0016)  when  the  FY04-05  group  was  compared  with  the 
initial  FY96-97  group.  The  design-build  total  project  time  showed  significant 
improvement  for  FY02-03  (p=0.035)  and  FY04-05  (p=0.013)  when  compared  to  their 
previous  respective  year  groups. 


The  historical  analysis  did  not  show  a  consistent  trend  with  the  total  project  time 
performance  of  the  traditional  method.  Although  the  FY04-05  year  group  (2.83  years) 
had  better  performance  than  the  starting  FY96-97  year  group  (3.05  years),  the 
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improvement  was  not  significant.  The  traditional  method  showed  significant  p=0.041 
worse  performance  for  the  FY02-03  year  group  from  the  previous  FY00-01  year  group. 
Therefore,  the  recovery  for  the  FY04-05  year  group  was  significant  (p=0.022)  from  the 
FY02-03  performance.  The  t-test  results  for  the  time  comparison  of  total  project  time  are 
provided  in  Appendix  J. 

The  most  interesting  result  showed  the  FY02-03  and  FY04-05  design-build  year 
groups  (126  projects)  outperformed  the  performance  of  the  traditional  method  (165 
projects),  though  not  at  a  significant  level.  The  total  project  time  is  affected  by  the 
previously  discussed  construction  timeline.  The  design-build  construction  timeline 
outperformed  the  traditional  method  and  directly  resulted  in  improving  the  total  project 
time.  The  historical  analysis  shows  that  the  design-build  method  is  now  able  to  start  and 
complete  MILCON  projects  in  shorter  total  project  time. 

Facility  Type  Analysis 

The  final  analysis  of  this  study  will  attempt  to  answer  if  the  design-build  delivery 
method  is  better  suited  for  a  particular  facility  type.  Nine  facility  types  emerged  from  the 
sample  of  835  projects:  airfield  pavements,  operations,  maintenance,  corrosion  control, 
storage,  administration,  dormitory,  fitness  center,  and  child  development  center.  Table 
14  presents  the  delivery  method  that  had  the  better  perfonnance  for  the  corresponding 
metric  and  facility  type.  Results  that  showed  a  NA  lack  a  sufficient  sample  size  to 
conduct  an  analysis.  Before  the  results  are  presented,  a  brief  explanation  of  how  the  nine 
facility  types  were  chosen  is  required. 
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Table  14.  Analysis  Result  Summary 
(Method  with  better  metric  performance  shown) 


Performance  Metric  J 

Unit 

Cost 

Cost 

Growth 

Schedule 

Growth 

Construction 

Speed 

Modifications 
Per  $M 

CWE/PA 

Construction  Timeline 

lotal 

Project 

Time 

LO 

V 

< 

Q_ 

o 

CN 

V 
< 
Q_ 

V 

LO 

o 

CN 

A 

< 

O. 

Overall  Results 

DB*** 

DB  ** 

DB 

DB*** 

DB  ** 

DB 

T 

J  ** 

T 

Facility  Type  (CAT  CODE) 

Airfield 

Pavements 

(11XXXX) 

DB* 

T 

DB 

DB 

DB  ** 

T 

NA 

T 

NA 

DB 

Operations 

(141XXX) 

T 

DB  ** 

T 

T 

DB  ** 

T 

DB 

T 

NA 

T 

Maintenance 

(21XXXX) 

T 

DB 

DB 

T 

DB  ** 

DB 

T 

J  ** 

NA 

T 

Corrosion 

Control 

(211159) 

DB 

DB* 

T 

DB 

DB  ** 

DB 

NA 

DB 

NA 

T* 

Storage 

(442758) 

DB 

DB 

T 

DB 

DB 

DB 

T 

T 

NA 

DB 

Administration 

(6102XX) 

DB 

DB 

DB 

DB* 

DB* 

DB 

NA 

T 

NA 

T 

Dormitory 

(721312) 

T 

DB* 

DB 

DB* 

DB 

NA 

T 

NA 

T 

Fitness  Center 
(740674) 

DB* 

DB 

DB 

DB 

DB* 

T 

DB 

T 

NA 

T 

Child 

Development 

Center 

(740884) 

DB 

DB  ** 

DB* 

DB 

DB 

DB 

DB 

T 

NA 

T 

DB  =  Design  Build 
T  =  Traditional 
NA  =  Not  Applicable 

*  =  Significant  (p<=0.05  one-tail) 

**  =  Highly  Significant  (p<=0. 01  one-tail) 

***  =  After  consideration  of  facility  type 
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Facilities  were  chosen  for  analysis  according  to  the  minimum  sample  size  of  five 
projects  per  delivery  method  as  outlined  in  the  methodology.  Facility  categorization 
depended  on  the  primary  Category  Code  (Cat  Code)  determined  by  the  first  two  numerals 
in  the  six  digit  code.  Facility  type  was  further  refined  if  there  were  sufficient  samples  to 
test  a  specific  building  type.  For  example,  although  within  the  maintenance  Cat  Code, 
corrosion  control  facilities  were  analyzed  separately  due  to  a  sufficient  sample  size  and 
the  project  characteristics  of  additional  HVAC  and  environmental  requirements  observed 
in  corrosion  control  facilities.  The  facilities  chosen  cover  the  majority  of  the 
recommended  facilities  for  design-build  use  recommended  by  the  Blue  Book  shown  in 
Table  8. 

Two  metrics,  unit  cost  and  construction  speed,  were  previously  analyzed  using  all 
facility  types  and  produced  inconclusive  results.  The  difficulty  stemmed  from  comparing 
these  metrics  with  varying  facility  types.  The  facility  type  analysis  successfully  analyzed 
unit  cost  and  construction  speed  metrics,  identifying  design-build  as  the  method  with  the 
best  performance  for  these  metrics.  The  unit  cost  comparison  resulted  in  six  of  nine 
facility  types  favoring  design-build.  The  significant  results  of  design-build  were  for 
airfield  pavement  and  fitness  center  facilities.  Construction  speed  also  showed  design- 
build  as  better  for  six  of  the  nine  facility  types.  Design-build  had  significant  results  for 
administration  buildings  while  the  traditional  method  had  highly  significant  results  for 
dormitory  projects.  With  design-build  having  better  unit  cost  and  construction  speed 
performance  for  the  majority  of  facilities,  it  can  be  speculated  that  design-build  is  better 
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than  traditional  for  these  metrics.  The  significant  findings  for  each  of  the  nine  facility 
types  will  now  be  discussed. 

Airfield  Pavements  (11XXXX) 

Airfield  pavements  include  runways,  taxiways,  parking  ramps,  and  cargo  pads. 
Table  15  displays  the  airfield  pavement  design-build  and  traditional  results  for  all 
performance  metrics.  The  method  with  the  best  airfield  pavement  performance  is  design- 
build.  Five  out  of  eight  perfonnance  metrics  had  better  design-build  perfonnance  with 
significant  (p=0.03 1)  unit  cost  and  highly  significant  (p=0.004)  Mods/$M.  Airfield 
pavements  were  only  one  of  two  facility  types  where  design-build  had  better  total  project 
time.  The  traditional  method  only  had  better  cost  growth,  CWE/PA,  and  construction 
timeline  but  not  at  a  significant  level.  Airfield  pavements  were  the  only  facility  type  in 


which  either  method  was  below  the  construction  timeline  goal. 


Table  15.  Airfield  Pavement  Facility  Analysis 


Performance  Metric 

1  Mean 

j  Variance 

j  Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

139.32495 

230.87844 

2210.0211 

46470.333 

5 

26 

0.0313169 

Cost  Growth(%) 

5.6189186 

4.0051904 

76.246093 

34.255572 

8 

40 

0.3150306 

Schedule  Growth  (%) 

10.332994 

28.625669 

586.3347 

4547.3104 

9 

39 

0.0915185 

Construction  Speed  (SM/Month) 

6390.0237 

6213.8585 

6762561.8 

96331310 

5 

26 

0.4690931 

Modifications  Per  $M  (#/$M) 

1.0584273 

3.3531247 

1.5101549 

20.80633 

8 

40 

0.0046627 

CWUPA 

0.9435221 

0.8908499 

0.0180473 

0.0413035 

9 

40 

0.1763216 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

NA 

$5M  <  PA  <  $20M 

-49.833333 

-77.666667 

38995.767 

73323.433 

6 

21 

0.392912 

PA  >  $20M 

NA 

j  Total  Project  Time  (Years) 

2.3208904  |  2.8009589|  0.61 12118|  1.5993951 1  8|  40|  0.0898772 

Operations  (141XXX) 


The  operations  facilities  analyzed  consisted  of  base  operations  and  squadron 


operations  facilities.  The  design-build  delivery  method  showed  to  have  the  best 


performance  for  operations  facilities.  Design-build  had  a  better  perfonnance  at  a  highly 
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significant  level  for  cost  growth  (p=0.006)  and  Mods/$M  (p=0.0003).  Although  the 
traditional  method  had  better  perfonnance  for  six  out  of  nine  metrics,  none  of  the 
traditional  perfonnance  results  were  significant.  Table  16  outlines  the  operations 
facilities  delivery  method  results  for  all  performance  metrics. 


Table  16.  Operations  Facility  Analysis 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

2883.9348 

2461.0581 

1022902.7 

707755.89 

14 

49 

0.0849918 

Cost  Growth(%) 

2.327506 

5.2201591 

11.696428 

23.965746 

15 

54 

0.006747 

Schedule  Growth  (%) 

17.29903 

13.523083 

756.8929 

1197.8378 

15 

52 

0.3314962 

Construction  Speed  (SM/Month) 

150.97182 

168.3225 

4864.6346 

6170.7828 

14 

49 

0.2166755 

Modifications  Per$M  (#/$M) 

2.4137192 

5.3940911 

4.9734443 

17.889993 

15 

57 

0.0002953 

CWE/PA 

1.0138376 

0.9553369 

0.0138806 

0.0177626 

15 

57 

0.0546267 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

189.25 

190.52381 

62004.917 

46539.662 

m 

21 

0.4964114 

$5M  <  PA  <  $20M 

209.63636 

102.02941 

53136.855 

58413.605 

ii 

34 

0.1001066 

PA  >  $20M 

NA 

Total  Project  Time  (Years) 

3.3623744 

3.1173276 

0.6935136 

1.1013913 

15 

57 

0.1735139 

Maintenance  (21XXXX) 

The  maintenance  facility  Cat  Code  includes  aircraft  maintenance  hangars, 
munitions  maintenance,  missile  maintenance,  aerospace  ground  equipment  (AGE) 
facilities,  and  civil  engineer  complexes.  Table  17  displays  the  results  for  the  analysis  of 
maintenance  facilities  for  all  performance  metrics.  Neither  delivery  method  emerged  as 
the  best  method  to  use  for  maintenance  facilities.  The  traditional  method  had  a 
significant  advantage  (p=0.016)  for  construction  timeline  with  five  of  nine  metrics  in 
favor  of  traditional.  The  design-build  had  a  highly  significant  advantage  (p=4.7xl0‘5)  for 
Mods/$M.  The  delivery  methods  were  evenly  matched  for  both  cost  and  schedule 
metrics. 
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Table  17.  Maintenance  Facility  Analysis 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

3004.2297 

2492.6018 

1981022.2 

2181257.2 

27 

72 

0.0592417 

Cost  Growth(%) 

4.2015111 

6.0623967 

42.824887 

66.685865 

32 

88 

0.1015177 

Schedule  Growth  (%) 

14.261267 

15.112741 

468.12097 

801.12423 

31 

84 

0.4321463 

Construction  Speed  (SM/Month) 

207.79207 

512.74172 

27031.168 

7542839.4 

26 

72 

0.1758116 

Modifications  Per$M  (#/$M) 

1.7359121 

4.6090279 

4.7971254 

31.179345 

32 

88 

4.689E-05 

CWE/PA 

0.9657319 

0.9942665 

0.011919 

0.0279118 

32 

88 

0.1401677 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

219.875 

138.94 

69540.411 

19927.69 

8 

50 

0.2103203 

$5M  <  PA  <  $20M 

249.42857 

128.02778 

36157.757 

46599.399 

21 

36 

0.0160418 

PA  >  $20M 

NA 

Total  Project  Time  (Years) 

3.4272205 

3.1872864 

1.5896911 

2.0787518 

31 

89 

0.1916874 

Corrosion  Control  (211159) 

Corrosion  control  facilities  are  typically  the  paint  shops  for  aircraft  and 
equipment.  The  design-build  delivery  method  was  identified  as  the  best  method  for 
constructing  corrosion  control  facilities.  The  design-build  method  outperformed 
traditional  in  six  of  eight  metrics  including  a  significant  result  (p=0.038)  for  cost  growth 
and  a  highly  significant  result  (p=0.0063)  for  Mods/$M.  The  traditional  method  did 
report  a  significant,  almost  highly,  result  (p=0.0103)  for  total  project  time.  The  results 
for  corrosion  control  facilities  are  shown  in  Table  18. 


Table  1! 

$.  Corrosion  Coni 

trol  Facility  Analysis 

Performance  Metric 

Mean 

j  Variance 

|  Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

3182.0401 

3672.0032 

1638199.9 

4289100.3 

8 

9 

0.2811299 

Cost  Growth(%) 

3.852818 

14.220042 

15.154935 

225.04865 

8 

9 

0.0383452 

Schedule  Growth  (%) 

12.063306 

0.3709418 

968.18766 

96.405544 

8 

7 

0.1701015 

Construction  Speed  (SM/Month) 

229.38628 

161.86725 

3004.1391 

10225.875 

8 

9 

0.0530427 

Modifications  PerSM  (#/$M) 

1.2793511 

5.7254119 

0.8245366 

17.59159 

8 

9 

0.0063668 

CWE/PA 

1.0030469 

1.0591324 

0.0029169 

0.0152497 

8 

9 

0.1211001 

Constiuction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

NA 

$5M  <  PA  <  $20M 

154.33333 

172.5 

70446.267 

11867 

6 

4 

0.442575 

PA  >  $20M 

NA 

Total  Pro  ject  Time  (Years) 

3.7027397 

2.6885845 

0.7578951 

0.4830981 

8 

9 

0.0103447 
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Storage  (442758) 


Storage  facilities  include  base  supply,  logistics,  and  aircraft  parts  warehouses. 
Table  19  shows  the  results  for  the  storage  facility  analysis.  Neither  the  traditional  or 
design-build  method  proved  to  be  better  for  building  storage  facilities.  Neither  method 
produced  statistically  significant  results.  One  notable  fact  was  that  storage  facilities  were 
second  out  of  two  facility  types  where  the  design-build  method  had  a  faster  total  project 
time. 


Table  19.  Storage  Facility  Analysis 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

1722.411 

1993.6895 

721629.03 

862513.79 

11 

7 

0.272059 

Cost  Growth(%) 

3.5547324 

4.4827889 

17.797769 

52.192262 

11 

9 

0.3695856 

Schedule  Growth  (%) 

28.005109 

13.948306 

1318.6033 

1048.227 

10 

8 

0.1993889 

Construction  Speed  (SM/Month) 

864.30045 

249.79005 

4735088.6 

31857.49 

11 

7 

0.1867185 

Modifications  PerSM  (#/$M) 

2.4671865 

3.0861676 

3.8765671 

5.8789354 

11 

9 

0.2731644 

CWE/PA 

0.9915245 

0.9917386 

0.0092843 

0.0307953 

11 

9 

0.4987193 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

302.8 

146.28571 

96789.2 

17881.905 

5 

7 

0.1693738 

$5M  <  PA  <  $20M 

158.16667 

33.5 

24593.367 

924.5 

6 

2 

0.0572175 

PA  >  $20M 

NA 

Total  Project  Time  (Years) 

2.7668742 

2.8753425 

0.6338637 

0.8736069 

11 

10 

0.389499 

Administration  (6102XX) 

Administration  facilities  consist  of  headquarters  and  mission  support  buildings. 
The  design-build  delivery  method  was  identified  as  the  best  method  for  constructing 
administration  facilities.  The  design-build  method  outperformed  traditional  in  six  of 
eight  metrics  with  two  significant  results  for  construction  speed  (p=0. 1 05)  and  Mods/$M 
(p=0.026).  The  traditional  method  only  outperformed  design-build  in  construction 
timeline  and  total  project  time  but  not  at  a  significant  level.  The  results  for 
administration  facilities  are  shown  in  Table  20. 
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Table  20.  Administration  Facility  Analysis 


|  Performance  Metric 

;  Mean 

j  Variance 

|  Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

2142.4217 

2692.3414 

1009936.3 

1804155.6 

11 

20 

0.1043814 

Cost  Growth(%) 

9.4653069 

11.198354 

100.59537 

157.54746 

13 

23 

0.3266387 

Schedule  Growth  (%) 

8.1122765 

17.577922 

202.78299 

1433.1791 

13 

22 

0.1504258 

Construction  Speed  (SM/Month) 

344.10709 

190.40431 

30302.127 

17202.293 

11 

20 

0.0105631 

Modifications  Per  $M  (#/$M) 

3.086033 

7.8382245 

7.3799829 

114.32712 

13 

23 

0.0267426 

CWUPA 

0.9795442 

1.0114323 

0.0153172 

0.0212294 

13 

23 

0.2460941 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

NA 

$5M  <  PA  <  $20M 

197.18182 

134.53846 

62827.364 

46444.103 

11 

13 

0.2615021 

PA  >  $20M 

NA 

Total  Project  Time  (Years) 

3.6638567 

3.0986301 

1.6809387 

1.4915935 

13 

23 

0.1059119 

Dormitory  (721312) 

The  analysis  of  delivery  method  performance  for  dormitories  revealed  the  design- 
build  method  out-performed  the  traditional  method.  The  design-build  method  had  two 
significant  results  for  cost  growth  (p=0.023)  and  Mods/$M  (p=0.034)  while  the 
traditional  method  only  had  one  significant  result.  Traditional  projects  perfonned  at  a 
highly  significant  (p=0.0021)  level  for  construction  speed.  The  dormitory  analysis  used 
the  number  of  rooms  as  the  scope  for  the  unit  cost  and  construction  speed  metrics.  Table 
2 1  shows  the  results  for  the  dormitory  facility  analysis. 


Table  21.  Dormitory  Facility  Analysis 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/Room) 

93595.204 

90128.321 

1.317E+09 

1.083E+09 

31 

39 

0.3403111 

Cost  Growth(%) 

3.0037286 

6.3382485 

7.635004 

103.26691 

36 

42 

0.0234171 

Schedule  Growth  (%) 

13.698112 

20.881137 

538.50341 

1135.3679 

36 

37 

0.1458431 

Construction  Speed 

5.6231229 

7.893082 

4.151523 

17.456074 

31 

39 

0.0021224 

Modifications  PerSM  (#/$M) 

2.096505 

3.0727112 

5.6856014 

4.8556201 

35 

42 

0.0341728 

CWE/PA 

1.003325 

1.0155387 

0.0095174 

0.0175088 

36 

42 

0.3205994 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

NA 

$5M  <  PA  <  $20M 

161.22857 

116.08458 

53827.476 

37153.102 

35 

33 

0.1924834 

PA  >  $20M 

NA 

Total  Pro  ject  Time  (Years) 

3.6539335 

3.3767986 

1.1717492 

1.1847483 

35 

41 

0.135421 
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Fitness  Center  (740674) 

Table  22  displays  the  results  for  the  analysis  of  fitness  centers.  The  analysis 
identified  the  design-build  method  as  best  suited  for  fitness  centers.  Design-build 
outperformed  the  traditional  method  in  six  out  of  nine  metrics  with  significant  results  for 
unit  cost  (p=0.035)  and  Mods/$M  (p=0.0109). 


Table  22.  Fitness  Center  Facility  Analysis 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

2354.8515 

2990.1525 

628453.6 

1195255.1 

13 

18 

0.0354141 

Cost  Growth(%) 

4.2945129 

4.3558831 

15.575598 

18.986006 

13 

18 

0.4838434 

Schedule  Growth  (%) 

9.916603 

19.82723 

642.00091 

1432.6726 

13 

17 

0.1992958 

Construction  Speed  (SM/Month) 

215.01477 

165.07767 

5568.2199 

14777.821 

13 

18 

0.0843641 

Modifications  PerSM  (#/$M) 

1.9426814 

3.855718 

1.1169509 

9.2826648 

13 

18 

0.0109572 

CWE/PA 

1.0235286 

1.0037189 

0.010777 

0.0413937 

13 

18 

0.3629852 

Construction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

101.5 

109.5 

2244.5 

7701.1 

2 

6 

0.4391728 

$5M  <  PA  <  $20M 

134.45455 

118.25 

30861.473 

18381.477 

11 

12 

0.404141 

PA  >  $20M 

NA 

Total  Project  Time  (Years) 

3.1774499 

3.0456621 

0.657662 

0.7210491 

13 

18 

0.3325353 

Child  Development  Center  (740884) 

The  child  development  center  facility  type  demonstrated  the  most  success  by  the 
design-build  method  out  of  all  facilities  studied.  Seven  out  of  nine  metrics  resulted  with 
the  design-build  method  outperfonning  traditional  with  a  highly  significant  level 
(p=0.0073)  for  cost  growth  and  a  significant  level  (p=0.04)  for  schedule  growth.  The 
results  for  the  child  development  center  analysis  are  shown  in  Table  23. 
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Table  22 


Child  Development  Center  Analysis 


Performance  Metric 

Mean 

Variance 

Observations 

P(T<=t) 

one-tail 

DB 

T 

DB 

T 

DB 

T 

Unit  Cost  ($/SM) 

2890.9641 

3231.2856 

380515.26 

279828.72 

5 

11 

0.1604834 

Cost  Growth(%) 

3.3564487 

6.8573174 

3.1376262 

10.151813 

5 

11 

0.00735 

Schedule  Growth  (%) 

4.5672122 

15.366163 

3.453981 

376.96338 

4 

12 

0.0408063 

Construction  Speed  (SM/Month) 

134.4259 

105.3184 

2645.692 

2181.2128 

5 

11 

0.1581336 

Modifications  PerSM  (#/$M) 

3.3064026 

8.2833807 

3.5401642 

91.884632 

5 

12 

0.0544918 

CWE/PA 

1.0660252 

1.1348042 

0.0065795 

0.0052878 

5 

12 

0.072394 

Cons  tmction 

Timeline 
(#  days  over 
target) 

PA  <  $5M 

178.5 

229.66667 

21012.5 

11203.5 

2 

9 

0.3596243 

$5M  <  PA  <  $20M 

81.333333 

0 

6608.3333 

18657 

3 

3 

0.2203744 

PA  >  $20M 

NA 

Total  Project  Time  (Years) 

3.2515068 

2.8614155 

2.2702586 

0.2147992 

5 

12 

0.3002533 

Summary 

This  study  gathered  835  (557  traditional,  278  design-build)  military  construction 
projects  from  ACES-PM  to  quantitatively  detennine  the  best  delivery  method.  Projects 
were  compared  over  eight  performance  metrics.  The  design-build  method  out-performed 
design-bid-build  for  six  metrics:  unit  cost,  cost  growth,  schedule  growth,  construction 
speed,  modifications  per  million  dollars,  and  CWE/PA.  The  design-build  method  yielded 
highly  significant  results  for  cost  growth  and  number  of  modifications  per  million  dollars. 
The  traditional  method  out-performed  the  design-build  method  for  the  remaining  two 
metrics:  construction  timeline  and  total  project  time  at  a  highly  significant  level.  The 
historical  analysis  revealed  that  the  design-build  method  has  improved  significantly  for 
cost  growth,  modifications  per  million  dollars,  construction  timeline,  and  total  project 
time.  The  traditional  method  also  experienced  significant  improvement  for  the  cost 
growth  and  modifications  per  million  dollars  metrics.  Additionally,  design-build  has 
overtaken  the  traditional  method  in  construction  timeline  and  total  project  time.  Finally, 
the  facility  type  analysis  revealed  that  the  design-build  method  was  best  suited  for  seven 
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of  the  nine  facility  types:  airfield  pavements,  operations,  corrosion  control, 
administration,  dormitories,  fitness  centers,  and  child  development  centers.  Discussion 
and  conclusions  based  on  of  the  results  of  this  analysis  are  contained  in  the  next  chapter. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

This  chapter  summarizes  the  results  and  answers  the  research  questions  of  this 
thesis.  Where  Chapter  4  provided  the  results  of  the  raw  data,  this  chapter  will  provide 
insights  and  observations  made  from  comparing  the  AF  MILCON  design-build 
performance  with  design-bid-build.  First,  the  conclusions  made  from  this  research  will 
be  given  by  answering  the  research  questions.  The  significance  and  limitations  of  this 
study  will  then  be  identified.  Next,  recommendations  for  further  action  will  be 
suggested.  Finally,  the  recommendations  for  future  research  that  were  produced  from 
this  study  will  be  listed. 

Problem  Statement 

The  goal  of  this  thesis  is  to  use  an  empirical  approach  to  assess  if  the  design-build 
delivery  method  is  better  than  the  traditional  design-bid-build  method  for  Air  Force 
MILCON  projects.  It  will  determine  if  the  design-build  method  performs  better  for 
certain  facility  types.  The  results  will  be  analyzed  to  detennine  if  the  success  of  the 
design-build  method  has  improved  through  the  years. 

Research  Questions 

1.  Does  the  design-build  delivery  method  for  MILCON  result  in  better  cost 

performance  characteristics  than  the  traditional  design-bid-build  approach? 

The  cost  characteristics  compared  by  this  study  included  unit  cost,  cost  growth, 
and  C WE/PA  ratio  metrics.  The  design-build  method  out-performed  the  traditional 
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method  in  all  three  performance  metrics  using  all  the  projects  in  the  sample.  The  cost 
growth  of  design-build  was  better,  at  a  highly  significant  level,  than  the  cost  growth  of 
traditional  projects.  The  Air  Force  FY06  Dirtkicker  goal  is  to  keep  MILCON  projects 
under  5%  cost  growth  (Fox,  2006).  The  design-build  method  met  this  goal  while 
traditional  projects  did  not.  Although  closely  matched  and  not  statistically  significant, 
the  CWE/PA  for  design-build  was  better  than  the  traditional  method.  Since  the  CWE/PA 
comparison  did  not  reveal  any  difference,  it  may  be  “a  metric  that’s  done  its  job  and 
needs  to  be  retired  now”  (MAJCOM  MILCON  manager). 

Initially,  the  unit  cost  of  the  traditional  method  outperformed  design-build.  The 
unit  cost  results  were  inconclusive  until  the  facility  type  was  considered  for  the  unit  cost 
metric.  Once  facility  type  was  taken  into  account,  the  design-build  delivery  method 
resulted  with  better  unit  cost  in  six  of  nine  facilities  with  a  significant  level  for  airfield 
pavements  and  fitness  centers.  With  design-build  having  better  unit  cost  perfonnance  for 
the  majority  of  facilities,  it  can  be  speculated  that  design-build  has  better  unit  cost 
performance  than  traditional. 

The  cost  results,  specifically  the  cost  growth  metric,  show  the  benefits  built  into 
the  design-build  process.  Once  MAJCOMs  started  using  the  design-build  process,  the 
traditional  method  problem  of  change  orders  was  no  longer  an  issue.  “With  traditional, 
we  used  to  manage  by  change  order.  Not  positive  change  orders  that  are  getting  you 
more  facility.  They  are  things  the  contractors  finds  such  as  design  errors  and 
misinterpretations  in  drawings  that  the  government  bears  the  cost  for”  (MAJCOM 
MILCON  manager).  Having  a  single  entity  responsible  for  the  design  and  construction 
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of  a  MILCON  projects  has  removed  the  incentive  and  ability  of  contractors  to  seek  out 
change  orders  and  helps  reduce  cost  growth. 

2.  Does  the  design-build  delivery  method  for  MILCON  result  in  better  schedule 

performance  characteristics  than  the  traditional  design-bid-build  approach? 

The  schedule  performance  metrics  studied  were  schedule  growth,  construction 
speed,  three  construction  timeline  categories  based  on  PA,  and  total  project  time.  The 
design-build  delivery  method  yielded  better  results  for  only  two  of  the  six  metrics  for  the 
entire  sample  size.  Design-build  outperformed  design-bid-build  with  less  schedule 
growth,  but  was  not  significant.  As  with  unit  cost,  the  construction  speed  metric  was 
influenced  by  the  type  of  facility.  When  projects  were  compared  according  to  facility 
type,  design-build  had  a  faster  construction  speed  for  six  of  the  nine  facility  types  with  a 
significant  level  for  administration  facilities.  With  design-build  having  better 
construction  timeline  performance  for  the  majority  of  facilities,  it  can  be  speculated  that 
design-build  has  better  performance  than  traditional  for  the  construction  timeline  metric. 

The  traditional  delivery  method  yielded  a  shorter  construction  timeline  for  all 
three  PA  categories  and  shorter  total  project  time  than  design-build.  The  construction 
timeline  comparison  for  projects  valued  at  $5M  <  PA  <  $20M  resulted  in  the  traditional 
method  outperforming  design-build  at  a  highly  significant  level.  Additionally,  the 
traditional  method  had  a  highly  significant  shorter  total  project  time  than  the  performance 
of  design-build. 

The  results,  that  the  traditional  delivery  method  had  better  timeline  characteristics 
than  design-build,  were  not  surprising  to  AF  project  managers.  Although  design-build 
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RFP  only  takes  four  months  to  finish,  the  project  sits  on  the  shelf  until  award  while  the 
traditional  method  is  being  designed.  The  possible  explanation  is  based  on  the  AF 
MILCON  bureaucratic  process  which  holds  the  schedule  benefits  of  design-build  back. 
“It’s  because  of  the  nature  of  how  our  process  muddles  along  its  way  with  awarding 
construction  money.  If  we  were  able  to  get  construction  money  as  soon  as  we  were  ready 
to  award,  design-build  would  be  better.  Because  both  (methods)  reach  October,  then  we 
wait  till  the  money  comes,  design-bid-build  catches  up”  (MAJCOM  MILCON  manager). 
As  stated  in  Chapter  4,  the  key  milestone  in  the  MILCON  process  is  the  signing  of  the 
defense  bill  by  the  President  that  funds  the  MILCON  projects  for  that  fiscal  year.  The 
defense  bill  provides  the  MILCON  construction  funds  that  enable  projects  to  be  awarded. 
“Once  you  award  the  contract,  if  you  have  got  traditional,  the  guy  just  starts  building.  If 
you’ve  got  a  design-build,  you  take  anywhere  from  two  to  six  months  up  front  to  get  it 
designed  to  the  point  where  you’re  ready  to  start  building  and  then  you  still  have  to  build 
the  project”  (MAJCOM  MILCON  manager).  Although  the  results  for  the  entire  study 
sample  size  showed  the  traditional  method  to  have  a  shorter  construction  timeline  and 
total  project  time,  the  historical  analysis  conducted  to  answer  the  fifth  research  question 
yielded  significant  findings  for  these  performance  metrics. 

3.  Does  the  design-build  delivery  method  for  MILCON  result  in  fewer 

modifications  than  the  traditional  design-bid-build  approach? 

This  study  showed  that  the  design-build  delivery  method  had  less  project 
modifications  per  million  dollars  (Mods/$M)  than  the  traditional  method  at  a  highly 
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significant  level.  This  made  sense  to  AF  program  managers  because  design-build  does 
not  see  the  change  orders  from  design  omission,  errors,  or  interpretations  common  to  the 
traditional  method.  However,  modifications  have  different  meanings  depending  on  what 
delivery  method  is  being  used.  “For  design-build  projects,  modifications  are  used  to  add 
value  to  the  project.  Traditional  projects  are  managed  by  change  order  where 
modifications  are  used  to  fix  problems  with  the  project”  stated  a  MAJCOM  senior 
executive  involved  in  MILCON  execution.  Modifications  are  directly  tied  to  cost  and 
schedule  growth.  It  makes  sense  that  the  design-build  delivery  method  had  better 
modification  performance,  thus  having  better  cost  and  schedule  growth  performance. 

4.  Has  the  design-build  delivery  method  shown  a  statistically  significant 
increased  performance  level  over  the  traditional  design-bid-build  with 
regard  to  cost  and  schedule  measures? 

T-test  results  with  a  p-value  of  less  than  0.05  were  considered  significant  and 
highly  significant  with  a  p-value  of  less  than  0.01.  The  significant  findings  have  already 
been  described  in  the  performance  metric  discussion.  The  overall  data  analysis  resulted 
in  design-build  outperforming  traditional  projects  at  a  highly  significant  level  for  cost 
growth  and  modifications  per  million  dollars.  While  traditional  projects  outperfonned 
design-build  at  a  highly  significant  level  for  the  $5M  <  PA  <  $20M  construction  timeline 
and  total  project  time  metrics.  The  significant  results  discovered  in  the  historical  and 
facility  type  analysis  will  be  discussed  in  respective  sections  that  follow. 
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5.  Using  these  measures  of  success,  has  the  design-build  delivery  method 
improved  over  recent  years  at  a  statistically  significant  level? 

The  historical  analysis  revealed  that  the  design-build  method  has  improved 
significantly  for  cost  growth,  modifications  per  million  dollars,  construction  timeline,  and 
total  project  time  perfonnance  metrics.  Additionally,  the  design-build  method  has 
recently  outperformed  the  traditional  method  in  construction  timeline  and  total  project 
time.  Two  reasons  have  been  identified  as  the  possible  causes  for  the  improvement  in  the 
design-build  method  for  AF  MILCON.  First,  the  AF  gained  experience  using  the  design 
build  delivery  method.  Secondly,  the  institutionalizing  of  the  Dirtkicker  criteria  set  goals 
for  the  perfonnance  of  design-build  and  traditional  MILCON  projects. 

Air  Force  Design-Build  Experience 

The  literature  review  identified  that  design-build  performance  increases  as  the  learning 
curve  progress  is  achieved  (Mouritsen,  1993;  Thornburn,  1994)  and  as  the  project  team 
gains  experience  (Beard  et  ah,  2001 ;  Cushman  &  Loulakis,  2001,  Link  2006).  The  use  of 
design-build  by  the  AF  increased  from  18%  use  in  FY1999  rising  to  48%  for  FY2000. 
Figure  20  displays  the  percentage  of  design-build  used  by  the  MAJCOMs  for  FY1990 
and  FY2000.  Prior  to  FY2000,  six  of  the  nine  MAJCOMs  used  design-build  for  less  than 
10%  of  their  projects.  The  increase  in  use  coincided  with  an  increase  in  the  design-build 
construction  timeline  and  total  project  time  for  the  FY00-01  year  group.  The  worsening 
of  these  metrics  for  FY00-01  could  have  resulted  from  the  large  learning  curve  for  new 
MAJCOMs  using  design-build.  The  improvement  in  design-build  construction  timeline 
and  total  project  time  observed  after  FY2002  could  be  attributed  to  AF  program 
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managers  gaining  design-build  experience  after  progress  was  achieved  on  the  initial 
learning  curve. 


%  Design-Build  by  MAJCOM 


ACC  AETC  AFMC  AFRC  AFSOC  AFSPC  AMC  PACAF  USAFE 


Figure  20.  FY99  and  FY00  Design-Build  use  by  MAJCOM 
Effects  from  Dirtkicker  Criteria 

Project  managers  feel  that  the  institutionalizing  of  the  Dirtkicker  criteria  improved 
the  performance  of  design-build  and  traditional  MILCON  projects.  The  FY02-03  and 
FY04-05  year  groups  saw  consistent  improvement  for  the  design-build  method  in  the  cost 
growth,  construction  timeline,  and  total  time  performance  metrics.  It  was  in  2003  that 
Dirtkicker  goals  were  initiated  by  Major  General  Robbins  and  then  carried  on  by  Major 
General  Fox  during  their  tours  as  The  Air  Force  Civil  Engineer.  “The  improvement  (in 
design-build)  is  because  of  Dirtkicker  goals.  They  are  now  fairly  institutionalized. .  .and 
we  are  trying  very  hard  to  meet  Dirtkicker  goals”  (MAJCOM  MILCON  manager). 
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One  possible  reason  for  the  improvement  observed  for  construction  timeline 
stems  from  the  target  days  set  forward  by  Dirtkicker.  Prior  to  FY2002,  there  was  not  a 
timeline  goal  that  MILCON  projects  had  to  reach.  To  meet  Dirtkicker  goals,  the  target 
days  established  for  the  construction  timeline  are  now  written  into  design-build  RFP  and 
traditional  IFB  packages  (Langley,  2007c).  This  change  could  explain  how  close  the 
design-build  and  traditional  construction  timeline  performance  is  for  FY02-03  and  FY04- 
05  year  groups.  The  total  project  time  performance  metric  subsequently  benefited  from 
this  improvement  in  construction  timelines. 

The  cost  growth  metric  also  significantly  improved  over  time  for  design-build  and 
traditional  methods.  The  FY02-03  and  FY04-05  year  groups  showed  a  very  close  cost 
growth  performance  between  the  two  delivery  methods.  Again,  this  occurred  at  the  time 
of  the  initiation  and  acceptance  of  Dirtkicker.  Because  both  delivery  methods  were  being 
graded  by  the  same  cost  growth  standard,  MAJCOM  MILCON  managers  feel  it  would 
make  sense  that  the  possible  cause  for  this  improvement  over  time  was  the  reaction  to 
Dirtkicker  goals. 

The  historical  analysis  also  discovered  the  modifications  per  million  dollars 
(Mods/$M)  metric  significantly  improved  over  time  for  design-build  and  traditional 
delivery  methods.  Modifications  are  the  primary  cause  of  cost  and  schedule  growth.  The 
first  observation  for  this  metric  was  that  the  design-build  method  had  significantly  less 
Mods/$M  than  the  traditional  method.  The  second  observation  was  that  both  delivery 
method  Mods/$M  improved  at  a  significant  level.  MAJCOM  project  managers  again  feel 
that  institution  of  Dirtkicker  in  2003  was  the  possible  cause.  The  primary  way  to  meet 
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the  Dirtkicker  cost  and  schedule  growth  goals  was  to  reduce  the  number  of  modifications 
and  change  orders  that  impact  MILCON  projects.  The  design-build  and  traditional  FY04- 
05  year  group  experienced  significant  Mods/$M  improvement  from  the  previous  FY02- 
03  year  group. 

6.  What  facility  types  make  the  design-build  method  a  better  option  over  the 
traditional  design-bid-build  approach? 

The  facility  type  analysis  revealed  that  the  design-build  method  was  best  suited 
for  six  of  the  nine  facility  types:  airfield  pavements,  operations,  corrosion  control, 
administration,  fitness  centers,  and  child  development  centers.  Neither  delivery  method 
showed  an  overwhelming  advantage  for  the  remaining  three  facility  types:  maintenance, 
storage,  and  donnitories.  The  Blue  Book  (Department  of  the  Air  Force,  2000)  directs  AF 
project  managers  to  consider  facilities  that  use  private  sector  standards,  commercial 
standards,  and  AF  design  guides  as  appropriate  MILCON  projects  for  use  with  the 
design-build  method.  The  facility  analysis  confirmed  the  recommendation  made  by  the 
Blue  Book  to  use  design-build  for  flightline  facilities,  administration,  MWR  facilities, 
and  child  development  centers.  However,  the  Blue  Book  suggested  using  design-build 
for  hangars,  maintenance,  donnitories,  and  warehouses  in  which  the  study  did  not  show  a 
distinct  advantage  between  the  two  delivery  methods. 

Significance  of  Research 

This  thesis  provided  an  in-depth  empirical  analysis  of  AF  MILCON  after  the 
design-build  method  was  approved  for  widespread  use  and  a  sufficient  design-build  track 
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record  had  been  established.  The  research  was  conducted  using  one  of  the  largest  sample 
sizes  used  to  compare  the  design-build  (278)  and  traditional  (557)  delivery  methods. 

This  study  was  the  first  to  analyze  the  perfonnance  of  AF  MILCON  projects  over  time, 
using  Dirtkicker  metrics,  and  according  to  facility  type.  This  study  provides  empirical 
evidence  of  where  the  design-build  delivery  method  provides  an  advantage  to  the 
traditional  method  for  AF  MILCON  execution. 

Limitations  of  Research 

There  are  specific  limitations  associated  with  this  research.  There  are  several 
limitations  that  stem  from  the  reliance  on  project  data  from  ACES-PM.  Additional 
limitations  resulted  from  the  restrictions  placed  on  the  scope  of  the  study. 

First,  ACES-PM  replaced  the  Planning,  Design,  and  Construction  (PDC)  system 
used  to  manage  project  information  in  FY  2000  (Department  of  the  Air  Force,  2000). 
Data  retrieved  from  ACES-PM  for  projects  prior  to  FY00  had  to  be  transferred  between 
the  two  databases.  This  transfer  could  limit  the  available  infonnation  of  projects 
originally  stored  in  PDC.  Because  data  could  have  been  lost  in  the  transfer,  some  project 
information  could  have  been  altered. 

Secondly,  ACES-PM  was  designed  to  manage  project  information  for  design-bid- 
build  projects.  Fields  are  not  available  to  directly  compare  design-build  to  traditional 
projects.  Therefore,  design-build  project  milestones  were  forced  into  the  fields  available 
for  design-bid-build  projects.  The  NTP  to  BOD  field  for  design-build  included  design 
and  construction  time.  ACES-PM  did  not  capture  the  dates  that  the  design  was 
completed  and  construction  began.  For  traditional  projects,  the  NTP  to  BOD  field  only 
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captured  the  construction  portion  of  the  project.  The  design  milestones  only  documented 
a  design  start  date  was  captured  and  a  design  completion  date  was  not  documented.  As  a 
result,  this  restriction  on  metrics  using  NTP  to  BOD  skewed  the  results  of  in  favor  of 
traditional  projects.  The  construction  timeline  and  schedule  growth  metric  results 
required  the  reader  to  interpret  the  findings  with  an  understanding  that  NTP  to  BOD  for 
design-build  projects  include  design  and  construction  whereas  the  NTP  to  BOD  for 
traditional  projects  only  include  construction. 

Thirdly,  ACES-PM  MILCON  data  lacked  two  specific  details  that  would  have 
benefited  this  study.  ACES-PM  did  not  document  the  causes  of  modifications.  This 
study  was  limited  by  the  assumption  that  all  modifications  were  a  result  of  a  negative 
cause.  Additionally,  the  ACES-PM  database  does  not  distinguish  between  design-build 
variations.  Projects  that  are  by  definition  design-build,  fast-track,  or  bridging  are  all 
documented  in  ACES-PM  as  using  the  design-build  delivery  method. 

Finally,  the  scope  of  this  research  was  limited  to  conducting  a  strict  empirical 
analysis  that  did  not  investigate  causality.  This  research  focused  on  the  quantitative 
questions  of  delivery  method  performance  without  the  investigation  as  to  why  the  results 
behaved  as  they  did.  Several  experts  were  asked  to  give  his  or  her  anecdotal  opinion  as 
to  the  cause  of  the  results.  These  expert  opinions  were  only  intended  to  gauge  the 
reactions  of  AF  project  managers  to  the  results  of  the  study. 

Recommendations  for  Action 

Three  recommendations  for  action  by  the  AF  civil  engineer  community  involve 
the  use  of  design-build,  ACES-PM  documentation,  and  Dirtkicker  criteria.  The  first 
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recommendation  is  to  increase  the  use  of  design-build  for  AF  MILCON,  which  is  already 
being  implemented.  However,  the  development  of  a  faster  award  process  for  design- 
build  procurement  is  needed  to  remove  the  shelf  time  design-build  projects  experience 
waiting  for  the  MILCON  appropriations  bill  to  be  signed.  Second,  the  ACES-PM  project 
fields  need  to  be  updated  to  better  document  and  manage  each  MILCON  delivery 
method.  Design-build  is  becoming  the  default  delivery  method  for  AF  MILCON 
execution  (Morrison,  2007a).  ACES-PM  should  document  the  milestones  specific  to  the 
method  being  used  for  each  MILCON  project.  This  would  provide  for  accurate 
management  of  the  delivery  methods  and  allow  for  a  valid  comparison  between  the 
traditional  and  design-build  method.  Finally,  the  CE  community  needs  to  clearly  define 
how  the  Dirtkicker  criteria  are  calculated.  The  current  criteria  leave  the  definition  of 
each  metric  vague  so  that  each  MAJCOM  calculates  the  metrics  differently.  A  concise 
equation  to  calculate  Dirtkicker  criteria  will  ensure  an  accurate  assessment  of  MILCON 
performance  for  AF  managers.  Also,  determine  which  Dirtkicker  metrics  are  really 
worth  tracking.  For  example,  current  metrics  such  as  the  CWE/PA  ratio  might  be  no 
longer  relevant. 

Recommendations  for  Future  Research 

Several  topics  have  emerged  from  the  study  of  this  research  that  would  benefit  the  Air 
Force  Civil  Engineer  community. 

1 .  An  analysis  of  procurement  method  for  AF  MILCON  design-build  projects.  The 
cited  study  by  El  Wardini  et  al.  (2006)  could  be  reproduced  and  investigated  for 
AF  MILCON  projects.  El  Wardini  et  al.  called  for  additional  research  in  this  area 
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due  to  the  study’s  small  sample  size.  This  topic  was  of  interest  for  this  research; 
however,  researching  procurement  method  (low  bid,  best-value,  qualifications 
based)  would  have  been  too  large  of  a  scope  to  include  in  this  study. 

2.  An  analysis  of  the  PIPS  performance  contracting  process  for  AF  MILCON. 
AFMC  is  testing  this  process  on  two  FY08  BRAC  pilot  projects  at  Kirtland  AFB 
with  an  estimated  completion  date  of  December  2009.  The  performance 
information  procurement  system  (PIPS)  process  was  developed  by  Dr.  Dean 
Kashiwagi  at  Arizona  State  University  (dean.kashiwagi@asu.edu)  as  a  new  delivery 
method  for  construction  projects  that  diverges  from  traditional  and  design-build 
practices.  The  manager  and  advocate  at  AFMC  is  Mr.  Douglas  Langley 
AFMC/A7CCC. 

3.  An  analysis  of  overseas  and  forward  deployed  project  delivery  methods  for 

MILCON.  This  and  previous  MILCON  studies  have  investigated  CONUS 
projects.  A  study  of  overseas  MILCON  has  not  yet  been  accomplished.  An 
investigation  in  overseas  (PACAF,  USAFE)  and  contingency  (Iraq,  Afghanistan) 
locations  would  be  extremely  interesting  and  beneficial  to  the  military 
engineering  community. 

4.  A  comparison  between  private  and  government  design-build  processes.  Standard 
policies,  practices,  contracts,  and  guides  have  been  published  by  the  DBIA,  AIA, 
and  ASCE  for  the  design-build  process.  These  standards  are  what  private 
industry  and  design-build  firms  use.  Additionally,  each  government  design 
agencies  (ACoE,  NAVFAC,  AFCEE)  and  AF  MAJCOM  manage  design-build 
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differently  with  their  own  guides  and  opinions.  Investigating  why  each  agency 
has  differing  opinions  and  developing  a  process  that  aligns  itself  with  standard 
practices  would  be  a  major,  yet  rewarding,  undertaking. 

5.  A  Decision  Analysis/Value  Focused  Thinking  (VFT)  analysis  of  design-build 
MILCON.  Detennining  what  selection  criteria  or  perfonnance  criteria  AF 
leaders,  project  managers,  and  customers  value  would  be  beneficial  in  order  to 
analyze  the  decision  to  use  design-build  or  traditional  delivery  for  MILCON. 

VFT  is  a  method  for  making  selection  decisions  that  has  not  been  looked  at  for 
use  with  MILCON. 

6.  Investigate  subjective  performance  criteria  for  design-build  vs.  design-bid-build 
MILCON  projects.  This  study  focused  on  the  objective  (time,  cost)  criteria  while 
the  subjective  (quality,  technical  performance,  functionality,  productivity, 
satisfaction,  and  environmental  sustainability)  criteria  are  just  as  important  to 
project  success  (Chan  et  ah,  2002).  Data  for  subjective  criteria  is  not  as  available 
in  ACES  as  objective  infonnation. 

7.  Environmental  sustainability  and  LEED  performance  of  design-build  MILCON 
projects.  Does  the  design-build  process  meet  more  LEED  criteria  and  produce 
more  innovative  solutions  to  environmental  concerns  than  design-bid-build? 
LEED  topics  are  appearing  in  more  design-build  publications  and  could  be 
investigated  for  MILCON. 

8.  An  investigation  of  design  agency  project  performance  over  time.  A  study  could 
be  conducted  using  Dirtkicker  criteria  to  investigate  which  design  agencies 
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(NAVFAC,  USAGE,  AFCEE)  have  better  MILCON  performance.  The  analysis 
could  investigate  which  districts  have  the  best  perfonnance.  Additionally,  a  study 
could  be  conducted  to  measure  how  MILCON  performance  has  changed  over 
time  comparing  AFCEE  management  with  previous  MAJCOM  management. 

This  could  be  conducted  once  AFCEE  has  established  a  track  record  for  managing 
all  of  the  AF  MILCON  execution. 

Summary 

This  chapter  described  the  conclusions  made  from  this  research  as  a  result  of 
answering  the  research  questions.  The  limitations  facing  this  study  were  identified  along 
with  the  significance  of  this  research.  As  a  result  of  this  research,  recommendations  were 
made  for  future  research  and  actions  that  could  be  taken  to  improve  the  management  of 
MILCON  projects.  This  study  provides  empirical  evidence  of  where  the  design-build 
delivery  method  provides  an  advantage  to  the  traditional  method  for  AF  MILCON 
execution. 
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Appendix  A:  ACES-PM  Project  Fields 


ACES-PM  Project  Description  Fields 


FIELD  NAME 

DESCRIPTION 

FY 

Fiscal  year  of  project 

Command  Req 

MAJCOM  requesting  project 

Installation 

Name  of  installation 

Project  Nbr 

Project  number  identification 

TITLE 

Title  of  project 

Project  Dsg  Method 

Traditional  or  Design-build  design  method 

Cat  Nbr 

Category  Code  of  facility 

Scope  Of  Project 

Scope  of  the  project 

Unit  Of  Measure 

Unit  of  measure  for  the  project  scope 

DSG  % 

%  design  complete 

CNS  % 

%  construction  complete 

ACES-P1 

VI  Project  Cost  Fields 

FIELD  NAME 

DESCRIPTION 

PA 

Programmed  Amount 

Total  Cwe  Am 

Total  Current  Working  Estimate  for  Project 

Mod  Count  Qy 

Number  of  modifications  to  the  project 

Contract  Mod  Am 

Cost  of  Modifications 

Contract  Orig  Am 

Original  awarded  contract  amount 

Contingency  Am 

Contingency  funds  amount 

Eng  Dsg  Am 

Project  design  cost  (traditional)  RPF  cost  (DB) 

Mgmt  Reserve  Am 

Management  and  reserve  amount 

Sioh  Am 

Supervision,  Inspection,  and  Overhead  amount 

Other  Cost  No  Sioh  Am 

Other  Cost  amount 

Other  Cost  Am 

Other  Cost  amount 

ACES-PM  Project  Description  Fields 


FIELD  NAME 

DESCRIPTION 

Act  Ramp  Dt 

Requirements  and  Management  Plan  (RAMP)  completion  date 

Field  Di  Issued 

Field  Design  Instruction  Issue  date 

Dsg  Start  Act 

Actual  design  start  date 

Act  Proj  Def  Approved  Dt 

Project  Definition  (PD)  approval  date 

Ready  To  Advertise  Act 

Ready  to  Advertise  date 

Di  Hq  Usaf  Auth  Advertise  Dt 

HQ  USAF  Authorization  to  Advertise  date 

Bid  Opening  Act 

Bidding  opening  date 

Di  Auth  To  Award  Dt 

Authorized  to  award  date 

Contract  Award  Act 

Contract  Award  date 

NTP 

Notice  to  Proceed  date 

NTP  to  ACT  BOD 

Number  of  Days  between  NTP  and  actual  BOD 

ESTBOD 

Estimated  Beneficial  Occupancy  Date 

ACT  BOD 

Actual  Beneficial  Occupancy  Date 

EST  CNS  COMPL 

Estimated  construction  complete  date 

ACT  CNS  COMPL 

Actual  construction  complete  date 

ACT  FC 

Actual  financial  closeout  date 
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Appendix  B:  Unit  Cost  Performance  over  Time  t-tests 


Design-Build  unit  cost  comparison  to  FY96-97  year  group 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

2237.973 

2964.186 

2932.062 

3705.406 

2661.744 

Variance 

1957556 

12523705 

2296017 

11505942 

1764085 

Observations 

13 

29 

51 

48 

36 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

40 

20 

49 

20 

tStat 

-0.95157 

-1.569384 

-2.348905 

-0.948569 

P(T<=t)  one-tail 

0.173515 

0.066123 

0.011453 

0.177081 

t  Critical  one-tail 

1.683851 

1.724718 

1.676551 

1.724718 

Design-Build  unit  cost  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY98-99 

FY00-01 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

2237.973 

2964.186 

2964.186 

2932.062 

2677.58 

3705.406 

3705.406 

2661.744 

Variance 

1957556 

12523705 

12523705 

2296017 

2709243 

11505942 

11505942 

1764085 

Observations 

13 

29 

29 

51 

27 

48 

48 

36 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

40 

34 

72 

65 

tStat 

-0.95157 

0.046519 

-1.762578 

1 .942358 

P(T<=t)  one-tail 

0.173515 

0.481584 

0.041108 

0.028214 

t  Critical  one-tail 

1.683851 

1 .690924 

1 .666294 

1 .668636 

Design-Build  vs.  traditional  unit  cost  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB  T 

DB  T 

DB  T 

DB  T 

DB  T 

Mean 

Variance 

Observations 

Hypothesized  Mean  Difference 
df 

tStat 

P(T<=t)  one-tail 
t  Critical  one-tail 

P(T<=t)  two-tail 
t  Critical  two-tail 

2237.97  2593.97 
1957556  2737960 

13  84 

0 

18 

-0.8318 

0.2082 

1.7341 

0.4164 

2.1009 

2964.19  2963.40 
12523705  6129025 

29  101 

0 

36 

0.0011 

0.4996 

1.6883 

0.9991 

2.0281 

2932.06  2166.77 
2296017  1833119 

51  65 

0 

101 

2.8282 

0.0028 

1.6601 

0.0056 

1.9837 

3705.41  3240.72 
11505942  4131705 

48  58 

0 

74 

0.8333 

0.2037 

1.6657 

0.4073 

1.9925 

2661.74  2772.96 
1764085  5286012 

36  41 

0 

65 

-0.2636 

0.3964 

1.6686 

0.7929 

1.9971 
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Appendix  C:  Cost  Growth  Performance  over  Time  t-tests 


Design-Build  cost  growth  comparison  to  FY96-97  year  group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

8.210679 

5.267267 

3.8238 

4.848114 

2.792385 

Variance 

65.38475 

53.57534 

19.76772 

44.31088 

22.92917 

Observations 

24 

46 

78 

77 

50 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

43 

27 

33 

31 

tStat 

1 .492552 

2.542191 

1.851078 

3.037021 

P(T<=t)  one-tail 

0.071428 

0.008532 

0.036563 

0.002407 

t  Critical  one-tail 

1.681071 

1.703288 

1.69236 

1.695519 

Design-Build  cost  growth  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY98-99 

FY0O-01 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

8.210679 

5.267267 

5.267267 

3.8238 

3.8238 

4.848114 

4.848114 

2.792385 

Variance 

65.38475 

53.57534 

53.57534 

19.76772 

19.76772 

44.31088 

44.31088 

22.92917 

Observations 

24 

46 

46 

78 

78 

77 

77 

50 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

43 

65 

132 

124 

tStat 

1 .492552 

1.212137 

-1.125078 

2.0216 

P(T<=t)  one-tail 

0.071428 

0.114924 

0.131299 

0.022685 

t  Critical  one-tail 

1.681071 

1.668636 

1.656479 

1.657235 

Design-Build  vs.  traditional  cost  growth  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB  T 

DB  T 

DB  T 

DB  T 

DB  T 

Mean 

8.211  8.264 

5.267  6.706 

3.824  7.267 

4.848  5.087 

2.792  3.286 

Variance 

65.385  129.199 

53.575  55.623 

19.768  1022.672 

44.311  56.963 

22.929  48.678 

Observations 

24  145 

46  143 

78  91 

77  99 

50  66 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

40 

77 

94 

171 

113 

tStat 

-0.028 

-1.154 

-1.016 

-0.223 

-0.451 

P(T <=t)  one-tail 

0.489 

0.126 

0.156 

0.412 

0.326 

t  Critical  one-tail 

1.684 

1.665 

1.661 

1.654 

1.658 

P(T<=t)  two-tail 

0.978 

0.252 

0.312 

0.824 

0.653 

t  Critical  two-tail 

2.021 

1.991 

1.986 

1.974 

1.981 

Traditional  cost  growth  comparison  between  year  groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY04-05 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

8.2638171 

3.285774 

7.266727 

5.086972 

5.086972 

3.285774 

Variance 

129.19915 

48.67806 

1022.672 

56.96294 

56.96294 

48.67806 

Observations 

145 

66 

91 

99 

99 

66 

Hypothesized  Mean  Different 

0 

0 

0 

df 

191 

99 

147 

t  Stat 

3.9008094 

0.634188 

1.571957 

P(T<=t)  one-tail 

6.637E-05 

0.263711 

0.059055 

t  Critical  one-tail 

1.6528705 

1.660391 

1.655285 

P(T<=t)  two-tail 

0.0001327 

0.527421 

0.11811 

t  Critical  two-tail 

1.9724619 

1.984217 

1.976233 
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Appendix  D:  Schedule  Growth  Performance  over  Time  t-tests 


Design-Build  schedule  growth  comparison  to  FY96-97  year  group 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

9.108851 

8.989413 

18.93575 

17.37141 

18.19836 

Variance 

342.0955 

749.2848 

718.708 

1003.431 

1570.508 

Observations 

24 

43 

77 

76 

49 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

62 

56 

68 

71 

tStat 

0.021221 

-2.023356 

-1.57684 

-1.335752 

P(T<=t)  one-tail 

0.491569 

0.02391 

0.059737 

0.092948 

t  Critical  one-tail 

1.669804 

1 .672522 

1 .667572 

1.6666 

Design-Build  schedule  growth  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY98-99 

FY00-01 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

9.108851 

8.989413 

8.989413 

18.93575 

18.93575 

17.37141 

17.37141 

18.19836 

Variance 

342.0955 

749.2848 

749.2848 

718.708 

718.708 

1003.431 

1003.431 

1570.508 

Observations 

24 

43 

43 

77 

77 

76 

76 

49 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

62 

85 

146 

86 

tStat 

0.021221 

-1.922771 

0.329523 

-0.122929 

P(T<=t)  one-tail 

0.491569 

0.028929 

0.371116 

0.451225 

t  Critical  one-tail 

1.669804 

1 .662979 

1.655357 

1 .662765 

Design-Build  vs.  traditional  schedule  growth  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 

FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

Mean 

9.11 

6.87 

8.99 

8.87 

18.94 

21.02 

17.37 

33.69 

18.20 

19.23 

Variance 

342.10 

556.33 

749.28 

592.73 

718.71 

1685.67 

1003.43 

1627.98 

1570.51 

1052.16 

Observations 

24 

121 

43 

129 

77 

88 

76 

99 

49 

62 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

39 

66 

151 

173 

92 

tStat 

0.516 

0.025 

-0.390 

-2.996 

-0.147 

P(T<=t)  one-tail 

0.304 

0.490 

0.349 

0.002 

0.442 

t  Critical  one-tail 

1.685 

1.668 

1.655 

1.654 

1.662 

P(T<=t)  two-tail 

0.609 

0.980 

0.697 

0.003 

0.884 

t  Critical  two-tail 

2.023 

1.997 

1.976 

1.974 

1.986 
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Appendix  E:  Construction  Speed  Performance  over  Time  t-tests 
Design-Build  construction  speed  comparison  to  FY96-97  year  group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

259.2114 

501.0464 

457.3444 

193.7201 

683.4642 

Variance 

44361.24 

2589962 

1422670 

11688.63 

3397020 

Observations 

13 

29 

51 

48 

35 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

30 

59 

14 

36 

tStat 

-0.794198 

-1.119773 

1.083143 

-1.338463 

P(T<=t)  one-tail 

0.216659 

0.133674 

0.148524 

0.094569 

t  Critical  one-tail 

1.697261 

1.671093 

1.76131 

1 .688298 

Design-Build  construction  speed  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY98-99 

FY00-01 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

259.2114 

501.0464 

501.0464 

457.3444 

457.3444 

193.7201 

193.7201 

683.4642 

Variance 

44361.24 

2589962 

2589962 

1422670 

1422670 

11688.63 

11688.63 

3397020 

Observations 

13 

29 

29 

51 

51 

48 

48 

35 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

30 

46 

51 

34 

t  Stat 

-0.794198 

0.127652 

1.571561 

-1.570038 

P(T<=t)  one-tail 

0.216659 

0.44949 

0.061118 

0.062834 

t  Critical  one-tail 

1.697261 

1.67866 

1.675285 

1.690924 

Design-Build  vs.  traditional  construction  speed  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB 

T 

DB  T 

DB 

T 

Mean 

259.2 

140.5 

501.0 

214.3 

457.3 

1242.3 

193.7  783.0 

683.5 

367.2 

Variance 

44361.2 

41807.4 

2589962.2 

151531.5 

1422669.5  26182500.0 

11688.6  10397217.3 

3397020.5 

506008.3 

Observations 

13 

80 

29 

100 

51 

65 

48  58 

35 

41 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

16 

29 

73 

57 

43 

t  Stat 

1.893 

0.951 

-1.196 

-1.391 

0.956 

P(T<=t)  one-tail 

0.038 

0.175 

0.118 

0.085 

0.172 

t  Critical  one-tail 

1.746 

1.699 

1.666 

1.672 

1.681 

P(T  <=t)  two-tail 

0.077 

0.349 

0.236 

0.170 

0.344 

t  Critical  two-tail 

2.120 

2.045 

1.993 

2.002 

2.017 
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Appendix  F:  Modifications  per  $M  Performance  over  Time  t-tests 

Design-Build  modifications  per  $M  comparison  to  FY96-97  year  group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

2.96536 

3.17173 

2.116389 

1.798117 

1.183972 

Variance 

4.161619 

30.93135 

4.247715 

2.77546 

2.807132 

Observations 

24 

46 

78 

77 

50 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

63 

39 

33 

38 

tStat 

-0.224393 

1.778525 

2.550501 

3.718143 

P(T<=t)  one-tail 

0.411589 

0.041557 

0.007791 

0.000323 

t  Critical  one-tail 

1 .669402 

1 .684875 

1.69236 

1 .685954 

Design-Build  modifications  per  $M  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97  FY98-99 

FY98-99  FY00-01 

FY00-01  FY02-03 

FY02-03  FY04-05 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Mean 

Variance 

Observations 

Hypothesized  Mean  Difference 
df 

tStat 

P(T<=t)  one-tail 
t  Critical  one-tail 

2.96536  3.17173 

4.161619  30.93135 

24  46 

0 

63 

-0.224393 

0.411589 

1 .669402 

3.17173  2.116389 
30.93135  4.247715 
46  78 

0 

52 

1 .237833 

0.110669 

1 .674689 

2.116389  1.798117 
4.247715  2.77546 

78  77 

0 

147 

1 .057957 

0.145905 

1 .655285 

1.798117  1.183972 
2.77546  2.807132 

77  50 

0 

104 

2.022715 

0.022834 

1 .659637 

Design-Build  vs.  traditional  modifications  per  $M  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 

FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

Mean 

2.96536 

5.705986 

3.17173 

6.117081 

2.116389 

4.587058 

1.798117 

3.799906 

1.183972 

2.233795 

Variance 

4.161619 

39.01531 

30.93135 

36.83073 

4.247715 

127.7004 

2.77546 

35.5635 

2.807132 

6.409343 

Observations 

24 

147 

46 

145 

78 

91 

77 

99 

50 

66 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

108 

82 

97 

117 

112 

tStat 

-4.137243 

-3.060072 

-2.04631 1 

-3.183979 

-2.6817 

P(T<=t)  one-tail 

3.49E-05 

0.001495 

0.021716 

0.000931 

0.004216 

t  Critical  one-tail 

1.659085 

1 .663649 

1.660715 

1 .657982 

1 .658573 

P(T<=t)  two-tail 

6.98E-05 

0.00299 

0.043432 

0.001862 

0.008433 

t  Critical  two-tail 

1.982173 

1.989319 

1 .984723 

1 .980448 

1.981372 

Traditional  modifications  per  $M  comparison  between  year  groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY04-05 

FY98-99 

FY00-01 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

5.705986 

2.233795 

6.117081 

4.587058 

4.587058 

3.799906 

3.799906 

2.233795 

Variance 

39.01531 

6.409343 

36.83073 

127.7004 

127.7004 

35.5635 

35.5635 

6.409343 

Observations 

147 

66 

145 

91 

91 

99 

99 

66 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

209 

123 

134 

142 

t  Stat 

5.766825 

1.188493 

0.592912 

2.31835 

P(T<=t)  one-tail 

1 .44E-08 

0.118464 

0.27712 

0.010929 

t  Critical  one-tail 

1.652177 

1.657336 

1.656305 

1.655655 

P(T<=t)  two-tail 

2.88E-08 

0.236929 

0.554239 

0.021858 

t  Critical  two-tail 

1.971379 

1.979439 

1.977826 

1.976811 
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Appendix  G:  C WE/PA  Performance  over  Time  t-tests 
Design-Build  C WE/PA  comparison  to  FY96-97  year  group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

1.017865 

0.979704 

0.997466 

0.971035 

1.004363 

Variance 

0.009281 

0.008852 

0.013917 

0.014772 

0.014567 

Observations 

24 

46 

78 

78 

50 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

46 

46 

48 

56 

tStat 

1.585756 

0.858109 

1.95111 

0.518537 

P(T<=t)  one-tail 

0.059823 

0.197641 

0.028447 

0.303063 

t  Critical  one-tail 

1.67866 

1.67866 

1 .677224 

1 .672522 

Design-Build  CWE/PA  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97  FY98-99 

FY98-99  FY00-01 

FY00-01  FY02-03 

FY02-03  FY04-05 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Mean 

Variance 

Observations 

Hypothesized  Mean  Difference 
df 

tStat 

P(T<=t)  one-tail 
t  Critical  one-tail 

1.017865  0.979704 
0.009281  0.008852 
24  46 

0 

46 

1.585756 

0.059823 

1 .67866 

0.979704  0.997466 
0.008852  0.013917 
46  78 

0 

111 

-0.922338 

0.179176 

1.658697 

0.997466  0.971035 
0.013917  0.014772 
78  78 

0 

154 

1.378153 

0.085078 

1.654808 

0.971035  1.004363 
0.014772  0.014567 
78  50 

0 

105 

-1.520041 

0.065753 

1 .659495 

Design-Build  vs.  traditional  CWE/PA  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 

FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

Mean 

1.017865 

0.987395 

0.979704 

1.013713 

0.997466 

0.967764 

0.971035 

0.98415 

1 .004363 

0.994009 

Variance 

0.009281 

0.055489 

0.008852 

0.091866 

0.013917 

0.031371 

0.014772 

0.027232 

0.014567 

0.027141 

Observations 

24 

147 

46 

145 

78 

91 

78 

99 

50 

66 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

78 

189 

158 

174 

114 

tStat 

1.10224 

-1.183356 

1 .29859 

-0.60855 

0.390614 

P(T<=t)  one-tail 

0.136873 

0.119077 

0.097988 

0.271808 

0.348406 

t  Critical  one-tail 

1 .664625 

1 .652956 

1 .654555 

1 .653658 

1 .65833 

P(T<=t)  two-tail 

0.273747 

0.238154 

0.195977 

0.543617 

0.696811 

t  Critical  two-tail 

1 .990847 

1 .972595 

1 .975092 

1 .973691 

1 .980992 
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Appendix  H:  Construction  Timeline  PA<$5M  Time  t-tests 


Design-Build  Construction  Timeline  PA<$5M  comparison  to  FY96-97  year  group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2  Variable  2  Variable  2  Variable  2 

Mean 

369.3333 

320 

166.7333 

172.2941 

56.07692 

Variance 

60407.5 

50688.67 

25364.35 

63458.22 

32377.08 

Observations 

9 

13 

15 

17 

13 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

16 

12 

17 

14 

t  Stat 

0.478918 

2.210166 

1.927982 

3.265482 

P(T<=t)  one-tail 

0.319238 

0.023633 

0.035366 

0.002818 

t  Critical  one-tail 

1.745884 

1.782288 

1.739607 

1.76131 

Design-Build  Construction  Timeline  PA<$5M  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97  FY98-99 

FY98-99  FY00-01 

FY00-01  FY02-03 

FY02-03  FY04-05 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Mean 

Variance 

Observations 

Hypothesized  Mean  Differenci 
df 

t  Stat 

P(T<=t)  one-tail 
t  Critical  one-tail 

369.3333  320 

60407.5  50688.67 

9  13 

0 

16 

0.478918 

0.319238 

1.745884 

320  166.7333 

50688.67  25364.35 

13  15 

0 

21 

2.049928 

0.026532 

1.720743 

166.7333  172.2941 

25364.35  63458.22 

15  17 

0 

27 

-0.07551 

0.470184 

1.703288 

172.2941  56.07692 

63458.22  32377.08 

17  13 

0 

28 

1.473185 

0.075927 

1.701131 

Design-Build  vs.  traditional  Construction  Timeline  PA<$5M  comparison  within 

year  groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

Mean 

369.3333 

172.557 

320 

186.8353 

166.7333 

125.2105 

172.2941 

182.4651 

56.07692 

63.63636 

Variance 

60407.5 

39097.63 

50688.67 

41973.23 

25364.35 

60326.71 

63458.22 

40038.64 

32377.08 

33321.74 

Observations 

9 

79 

13 

85 

15 

38 

17 

43 

13 

33 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

9 

15 

39 

24 

22 

t  Stat 

2.317929 

2.009148 

0.725185 

-0.14893 

-0.12777 

P(T<=t)  one-tail 

0.02282 

0.031435 

0.236334 

0.441426 

0.449745 

t  Critical  one-tail 

1.833113 

1.75305 

1.684875 

1.710882 

1.717144 

P(T<=t)  two-tail 

0.04564 

0.06287 

0.472667 

0.882852 

0.89949 

t  Critical  two-tail 

2.262157 

2.13145 

2.022691 

2.063899 

2.073873 

Traditional  Construction  Timeline  PA<$5M  comparison  between  year  groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97  FY04-05 

FY02-03  FY04-05 

Mean 

172.557  63.63636 

182.4651  63.63636 

Variance 

39097.63  33321.74 

40038.64  33321.74 

Observations 

79  33 

43  33 

Hypothesized  Mean  Difference 

0 

0 

df 

65 

72 

t  Stat 

2.807963 

2.697256 

P(T<=t)  one-tail 

0.003288 

0.004351 

t  Critical  one-tail 

1.668636 

1.666294 

P(T<=t)  two-tail 

0.006575 

0.008702 

t  Critical  two-tail 

1.997138 

1.993464 
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Appendix  I:  Construction  Timeline  $5M<PA<$20M  Time  t-tests 


Design-Build  Construction  Timeline  $5M<PA<$20M  comparison  to  FY96-97  year 

group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2  Variable  2 

Variable  2  Variable  2 

Mean 

185.1429 

165.1 

241.5833 

137.1404 

47.90625 

Variance 

20757.82 

96621.13 

61009.57 

77886.44 

19080.93 

Observations 

14 

30 

60 

57 

32 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

42 

33 

40 

24 

t  Stat 

0.292249 

-1.12892 

0.899307 

3.009849 

P(T<=t)  one-tail 

0.385767 

0.133537 

0.186936 

0.003031 

t  Critical  one-tail 

1.681952 

1.69236 

1.683851 

1.710882 

Design-Build  Construction  Timeline  $5M<PA<$20M  comparison  between  year 

groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97  FY98-99 

FY98-99  FY00-01 

FY00-01  FY02-03 

FY02-03  FY04-05 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Variable  1  Variable  2 

Mean 

Variance 

Observations 

Hypothesized  Mean  Difference 
df 

t  Stat 

P(T<=t)  one-tail 
t  Critical  one-tail 

185.1429  165.1 

20757.82  96621.13 

14  30 

0 

42 

0.292249 

0.385767 

1.681952 

165.1  241.5833 

96621.13  61009.57 

30  60 

0 

48 

-1.17493 

0.12291 

1.677224 

241.5833  137.1404 

61009.57  77886.44 

60  57 

0 

112 

2.13941 

0.017287 

1.658573 

137.1404  47.90625 

77886.44  19080.93 

57  32 

0 

86 

2.014202 

0.023557 

1.662765 

Design-Build  vs.  Traditional  Construction  Timeline  $5M<PA<$20M  comparison 


t-Test:  Two-Sample  Assuming  Unequal  Variances 

FY96-97 

within  year  groups 

FY98-99  FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB  T 

DB 

T 

DB 

T 

Mean 

185.1429 

109.0714 

165.1 

154.0144 

241.5833  67.17308 

137.1404 

118.0385 

47.90625 

37.64 

Variance 

20757.82 

66603.89 

96621.13 

74759.58 

61009.57  34350.58 

77886.44 

26230.19 

19080.93 

18025.24 

Observations 

14 

56 

30 

55 

60  52 

57 

52 

32 

25 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

37 

54 

108 

91 

52 

t  Stat 

1.47163 

0.163806 

4.258453 

0.441628 

0.28286 

P(T<=t)  one-tail 

0.074788 

0.435248 

2.2E-05 

0.329903 

0.389204 

t  Critical  one-tail 

1.687094 

1 .673565 

1.659085 

1.661771 

1.674689 

P(T<=t)  two-tail 

0.149576 

0.870496 

4.41  E-05 

0.659806 

0.778407 

t  Critical  two-tail 

2.026192 

2.004879 

1.982173 

1.986377 

2.006647 

Traditional  Construction  Timeline  $5M<PA<$20M  comparison  between  year 

groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY04-05 

FY02-03 

FY04-05 

Mean 

109.0714 

37.64 

118.0385 

37.64 

Variance 

66603.89 

18025.24 

26230.19 

18025.24 

Observations 

56 

25 

52 

25 

Hypothesized  Mean  Difference 

0 

0 

df 

77 

56 

t  Stat 

1.634298 

2.29669 

P(T<=t)  one-tail 

0.053139 

0.0127 

t  Critical  one-tail 

1.664885 

1.672522 
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Appendix  J:  Total  Project  Time  Performance  over  Time  t-tests 


Design-Build  total  project  time  comparison  to  FY96-97  year  group 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

Variable  1 

Variable  2 

Variable  2 

Variable  2 

Variable  2 

Mean 

3.619749 

3.413455 

3.541342 

3.19356 

2.781381 

Variance 

1.315913 

1.619136 

1 .562872 

1.280731 

0.837177 

Observations 

24 

45 

78 

77 

49 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

51 

41 

38 

38 

tStat 

0.68457 

0.286557 

1.594274 

3.126259 

P(T<=t)  one-tail 

0.248357 

0.387947 

0.05958 

0.001694 

t  Critical  one-tail 

1 .675285 

1.682878 

1.685954 

1 .685954 

Design-Build  total  project  time  comparison  between  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY98-99 

FY98-99 

FY00-01 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

3.619749 

3.413455 

3.413455 

3.541342 

3.541342 

3.19356 

3.19356 

2.781381 

Variance 

1.315913 

1.619136 

1.619136 

1.562872 

1 .562872 

1.280731 

1.280731 

0.837177 

Observations 

24 

45 

45 

78 

78 

77 

77 

49 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

df 

51 

91 

152 

117 

tStat 

0.68457 

-0.540335 

1.816157 

2.244678 

P(T<=t)  one-tail 

0.248357 

0.295143 

0.035658 

0.013334 

t  Critical  one-tail 

1 .675285 

1.661771 

1.65494 

1.657982 

Design-Build  vs.  traditional  total  project  time  comparison  within  year  groups 


t-Test:  Two-Sample  Assuming  Unequal  Variances 

FY96-97 

FY98-99 

FY00-01 

FY02-03 

FY04-05 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

DB 

T 

Mean 

3.619749 

3.056374 

3.413455 

3.188428 

3.541342 

2.869297 

3.19356 

3.244943 

2.781381 

2.830303 

Variance 

1.315913 

1 .729393 

1.619136 

1 .758024 

1 .562872 

1.895039 

1.280731 

2.57226 

0.837177 

1 .052073 

Observations 

24 

137 

45 

144 

78 

92 

77 

99 

49 

66 

Hypothesized  Mean  Difference 

0 

0 

0 

0 

0 

df 

35 

76 

167 

172 

109 

tStat 

2.169184 

1 .025083 

3.333861 

-0.248905 

-0.269202 

P(T<=t)  one-tail 

0.018473 

0.154288 

0.000528 

0.401865 

0.394142 

t  Critical  one-tail 

1 .689572 

1.665151 

1 .654029 

1.653761 

1.658953 

P(T<=t)  two-tail 

0.036946 

0.308576 

0.001055 

0.803731 

0.788283 

t  Critical  two-tail 

2.030108 

1.991673 

1.974271 

1 .973852 

1.981967 

Traditional  total  project  time  comparison  between  year  groups 

t-Test:  Two-Sample  Assuming  Unequal  Variances 


FY96-97 

FY04-05 

FY00-01 

FY02-03 

FY02-03 

FY04-05 

Mean 

3.056374 

2.830303 

2.869297 

3.244943 

3.244943 

2.830303 

Variance 

1.729393 

1.052073 

1.895039 

2.57226 

2.57226 

1.052073 

Observations 

137 

66 

92 

99 

99 

66 

Hypothesized  Mean  Difference 

0 

0 

0 

df 

161 

188 

163 

t  Stat 

1.337634 

-1.7405 

2.025093 

P(T<=t)  one-tail 

0.091451 

0.041703 

0.022245 

t  Critical  one-tail 

1.654373 

1.652999 

1.654256 

P(T<=t)  two-tail 

0.182902 

0.083407 

0.04449 

t  Critical  two-tail 

1.974808 

1.972663 

1.974625 
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Appendix  K:  Performance  Metric  Histograms 


Unit  Cost  Histograms 

DB  Unit  Cost  Histogram 
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Cost  Growth  Histograms 


Cost  Growth  Histograms  are  approximated  normal  due  to  a  cost  growth  starting  value  of 
zero. 


DB  Cost  Growth  Histogram 


□  Frequency 


Cost  Growth 


T  Cost  Growth  Histogram 


□  Frequency 


%  Growth 
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Schedule  Growth  Histograms 


DB  Schedule  Growth  Histogram 


□  Frequency 
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T  Schedule  Growth  Histogram 
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Construction  Speed  Histograms 


DB  Construction  Speed  Histogram 


□  Frequency 
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Modifications  per  Million  Dollars  Histograms 


DB  Mods/$M  Histogram 


□  Frequency 


Mods/$M 


T  Mods/$M  Histogram 
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CWE/PA  Ratio  Histograms 


DB  CWE/PA  Histogram 


□  Frequency 


CWE/PA  Ratio 


T  CWE/PA  Histogram 


□  Frequency 


CWE/PA  Ratio 
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Construction  Timeline  PA<$5M  Histograms 


DB  Construction  Timeline  PA<$5M 
Histogram 
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Construction  Timeline  $5M<PA<$20M  Histograms 


DB  Construction  Timeline 
$5M<PA<$20M  Histogram 


□  Frequency 


T  Construction  Timeline  $5M<PA<$20M 

Histogram 


□  Frequency 
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Construction  Timeline  PA>$20M  Histograms 


DB  Construction  Timeline  PA>$20M 
Histogram 
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Total  Project  Time  Histograms 


DB  Total  Project  Time  Histogram 


□  Frequency 


Years 


T  Total  Project  Time  Histogram 
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